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ABSTRACT 



Preferably a sensor receives a print image from an autho- 
rized person to form a template, and from a candidate to 
form test data. Power spectral density (PSD) data for the 
template and candidate are compared, to read out rotation & 
dilation; these are used to adjust the template or candidate 
preparatory to a correlation to find translation. After apply- 
ing the translation, and refinement of the rotation and 
dilation, normalized spatial correlation values (NSCVs) are 
used as a measure of quality of the match — and thresholded 
to make an early rejection or acceptance decision in very 
clear cases. Where the question is closer, isomorphic adjust- 
ment is applied to the entire template or candidate for a fairer 
comparison in their overlap area. Such comparison proceeds 
by the same type of PSD analysis — but for multiple subre- 
gions in the overlap area. Resulting NSCVs are averaged to 
obtain a measure of quality of the match, which again is 
thresholded for a final decision in the closer cases. Noise 
variance from the test data, vs. position in the image, is used 
to weight the importance of comparison with the template in 
each subregion. Nonvolatile memory holds instructions for 
automatic operation. 

22 Claims, 8 Drawing Sheets 
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ABSTRACT 



A method of analyzing and recognizing fingerprint images 
that utilizes vector processing of a vector field that is defined 
as the tangential vector of the fingerprint ridge curves is 
disclosed. The raw fingerprint image is divided into blocks, 
filtered to remove noise, and the orientation direction of each 
block is found. This allows the ridge curves to be enhanced 
and approximated by piece-wise linear approximations. The 
piece -wise linear approximations to the ridge curves allow 
the minutiae to be extracted and classified and a fingerprint 
minutiae template to be constructed. An enrollment process 
gathers multiple fingerprint images, creates fingerprint 
minutiae templates corresponding to the acquired fingerprint 
images, and stores the templates and other data associated 
with the respective individual or the enrolled fingerprint in 
a fingerprint database. In an identification process, an 
unknown raw fingerprint image is obtained via a fingerprint 
scanner and processed similarly to the enrollment process 
such that the fingerprint minutiae template of the unknown 
fingerprint is compared to one or more previously enrolled 
fingerprint minutiae templates. The identity of the individual 
associated with the unknown fingerprint is thereby ascer- 
tained. In addition, live finger detection can be accomplished 
in conjunction with the verification or identification process 
through analysis of the fingerprint image thus enhancing the 
security of the overall system. 
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FINGERPRINT NOTES 

Technical Information 



Image Resolution: 


Technical Specification 


False Rejection Rate (FRR): 




False Acceptance Rate (FAR): 


0.0001% ■■yiv-y. 


Allowable Fingerprint Rotation: ■> 


;+/-15degree ;'" : .>^s^ : :! ] '1a 


Operation Temperature: • : 


-25 to +65 Degrees Celsius MP 
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' Scanning Speed :4ife : y : ' : ' - - 
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Introduction 

ACTAtek ™ uses latest Optical Scanning technology with its own algorithms and matching calculations, a 
step above other sensors in the market. 

It must be emphasized that to get an accurate enrollment and quick authentication each time a fingerprint 
is presented, the fingerprint placement must be towards the center of the scanner. Placing your finger far 
from the center position of the sensor will increase the rejection rate. 

Finger Rotation should be kept to a minimum during enrollment and verification. 

When enrolling, place the finger on the sensor where the entire core can clearly be seen by the scanner. 

A good image is critical for the overall performance of the fingerprint scanner. Any deviation from a good 
image, either by placing the finger far away from the scanner, or by applying too much pressure or not 
locating it in the CENTER of the scanner, will cause the scanner's rejection rate to rise. Read below on how 
to get a good image for your enrollment/authentication. 

Good vs Bad Image 

A good fingerprint image is one in which the core of the fingerprint is well-defined and easily recognizable. 
Ibejc o re of a fi ng er i s def in ed as the "po i nt l ocate d w i th i n the i nner most recurving ridge", it is normally 
lo£atecUn_tb£^ It is therefore critical when enrolling that you place the finger on 

the scanner where the entire core can clearly be seen. 

An example of a good & bad image is displayed as follows: 




mm- 



Good Image: A reading Where the whole 
fingerprint core can be seen clearly. 



Bad Image: An image where the crackles & 
displacement of the fingerprint core decries it 
unrecognizable. 
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FINGERPRINT NOTES 



Fingerprint Enrollment & Authentication 

In order to receive a successful enrollment and authentication, it is critical that the following be noted 
carefully. Each successful enrollment will result in a successful authentication and save a lot of time in 
troubleshooting and erroneous readings. 



It is highly recommended for the fingerprint core be big and clear for a successful enrollment of a clear and good image. 

Make sure the fingerprint image captured is of the core of the finger presented. A fingerprint core is a point located 
within the innermost recurving ridge of any given finger. 

Also, to obtain a higher success rate, enroll the same finger 3 times in a slightly adjusted angle, one to the center, one 
inclined slightly to the left and the third inclined slightly to the right. 

If you follow the following enrollment procedure, the success rate will increase dramatically. 




> Optical Fingerprint Scanner 
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FINGERPRINT NOTES 



Fingerprint Enrollment: 

Step 1: Place the center of any one finger directly above the sensor in the dead center, as shown below: 



Step 2: Place the center of the same finger (enrolled in Step 1), slightly aligned to the left. 
Step 3: Place the center of the same finger, slightly aligned to the right. 

After each placement, wait for the message "Template Stored" on the LCD screen to appear, and then 
remove your finger and press "Enter/Return" to enroll the second or third finger(s). 

If you have any questions regarding the enrollment procedure, e-mail us at tech@hectrix.com . 




Singular Point Detection in Fingerprint Images Using 
Linear Phase Portraits 
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Abstract A fingerprint image represents a system of oriented texture, marked 
by a directed flow of ridges and valleys. The qualitative nature of this oriented 
flow is determined by the position and type of singular points such as the core 
and delta. The detection of the singular points and estimation of its position 
forms an important step in fingerprint image analysis and has several useful ap- 
plications such as classification, registration and orientation flow modeling. We 
present a novel approach for estimation of singular point position using linear 
phase portraits. Linear phase portraits are based on the geometric theory of dif- 
ferential equations and provide a qualitative description of vector flows. The 
proposed method has several advantages over existing schemes based on Poin- 
care index including robustness to noise. We present an objective evaluation of 
the algorithm using images from FVC2002 DB1 database. 



1 Introduction 

The fingerprint image represents a system of a strongly oriented texture consisting of a 
directed system of ridges and valleys. Minutiae features such as ridge ending and 
bifurcation mark local discontinuities in the ridge flow and constitute the local fea- 
tures. These features have been used extensively in fingerprint verification [5]. On the 
other hand, the orientation image and the singular points form the global features that 
give information about the coarser trends within the image. The orientation image 
captures the dominant direction of the ridge flow in each region of the fingerprint 
image. The singular points like 'core' and 'delta* mark the areas of interest within this 
flow. Figure 1 shows the local and global features of a typical fingerprint image. 
Among these features, the extraction of the singular points forms an important step in 
fingerprint image processing and has several useful applications. The classification of 
the ridge flow ('arch', 'tented arch'/whorl' or 'loop') can be directly determined based 
on the number, type and positions of the singular points. The singular points also 
represent the intrinsic points of reference within the fingerprint image and can there- 
fore be used to register or approximately align two different images during matching. 
Finally, there exist linear and non-linear methods [8,1 1,13] to accurately estimate the 
orientation of the fingerprint image given complete information about the singular 
points. However, estimating the location of the singular points itself is a difficult prob- 



lem. Existing methods are based on computing the Poincare" index of the orientation 
image at each pixel position [1,4]. These methods are sensitive to noise and generate 
many false positives in noisy images. In this paper, we present a robust algorithm 
based on linear phase portraits. Linear phase portraits are based on the geometric 
theory of differential equations and provide a qualitative description of vector flows 
and have traditionally used to qualitatively analyze vector fields arising from fluid 
flows. The technique cannot be directly applied to images due to inherent ambiguity in 
the direction of the edges. However, subsequent work by Rao and Jain[6,7], Shu et 
al[9], led to the development of several techniques that allow their use for analysis of 
oriented textures. In this paper, we further adapt the process for analysis of fingerprint 
images. 

The rest of the paper is organized as follows. Section 1 . 1 discusses the prior done in 
this area. Section 2 presents an overview of the proposed approach. The concept of 
linear phase portraits and its application to fingerprint images is discussed in detail in 
section 3. The results of the experiments are in Section 4. And finally, the conclusions 
are discussed in Section 5. 




<») (b) (S) (h) 



Figure 1 Illustration of fingerprint features. Global features: (b) Orientation image , singular 
points Core(c) and Delta (d). Local minutiae features: Bifurcation(e), Ridge dot(f), Ridge 
ending (g) and Ridge island(h). 

1.1 Prior Related Work 

There have been several approaches for the detection of singular points in literature. 
By far, the most popular method is the one proposed by Kawagoe and Tojo [4] and is 
based on the computation of the Poincar6 index. The Poincare" index for any given 
point in a vector field is computed by summing the orientation difference between 
successive elements along a closed path around that point. It can be shown that around 
closed curves the Poincare* index assumes only one of the discrete values 
0°,±180\±360*. The singular regions can be easily detected since the singularities 
corresponding to the loop, whorl and delta assume a Poincare" index value of 
180°,360° and -180° respectively . Recently, an interesting improvisation on the Poin- 
caire index was proposed by Bazen and Gerez [1]. In this method, the line integration 



around each point is avoided by using the result of Green's theorem which states that, 
a line integral around any point in a vector field can be replaced by a surface integral 
of the curl of the vector field [12]. However, methods based on Poincaire index are 
very sensitive to noise and lead to detection of false singularities in low-quality im- 
ages. 

Srinivasan and Murthy [10] proposed another approach based on the examination of 
the local characteristics of the orientation image. They examine the dominant direction 
of the orientation in four quadrants around each point and use several rules to detect 
and classify the singular points. This method is very heuristic in its approach. A more 
elegant approach was suggested by Capelli et al. [2]. They proposed a irregularity 
operator that indicates the local deviation of the orientation around each point. The 
operator is applied on each pixel in the image to yield an irregularity map. Singulari- 
ties can be extracted by examining the dark regions within the map. However, the 
extraction requires considerable processing of the irregularity map. They also pro- 
posed a novel partitioning based approach, where the ridge flow is segmented into 
regions of consistent orientation by an iterative clustering algorithm. Singular points 
are determined by considering the locations where the region borders intersect. 



2 Overview 
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Figure 2 Overview of the proposed approach 

Figure 2 shows the overview of the proposed approach. The orientation image describ- 
ing the ridge flow is obtained using well known methods outlined in [6]. While it is 
difficult to model the entire orientation image through any linear methods, it can be 
modeled as a combination of piece- wise linear flows. Each piece wise linear flow is 
obtained by moving a sliding window of size (WxW) across the orientation image. In 
our case W=7. At each position of the window, we derive a parametric representation 
of the local flow using linear phase portraits. This representation provides us not only 
with the possible location of the singularity, but also about the nature of the singular- 
ity. More details are presented in section 3. As each piece- wise linear flow is ana- 



lyzed, evidence is accumulated about the possible location and type of the singulari- 
ties in a spirit similar to generalized Hough transformation [3]. The two dimensional 
accumulator array can be treated as an intensity image, where each position indicates 
the likelihood of being a singular point. This image is processed using grey- scale 
morphology and connected component analysis to obtain a best estimate of the type 
and position of each of the singular points. The proposed approach of analyzing the 
orientation locally has several advantages over global fingerprint models such as 

1. The analysis assumes the orientation image to be piecewise linear, and can there- 
fore handle any arbitrary flow pattern 

2. The parameter estimation is done using a least square minimization approach 
rendering the process very robust to noise 

3. The analysis provides an over-complete description that can be used to recon- 
struct a smooth estimate of the ridge flow. 



3 Singular Point Detection 

In this section, we present an outline of linear phase portraits that is sufficient for 
analysis of fingerprint images and also explain its application to singular point detec- 
tion. A more complete discussion of linear phase portraits is presented in [6,7,9]. 

3.1 Linear Phase Portraits 



Phase portraits are based on the geometric theory of differential equations and are 
used to derive qualitative description of vector flows and fluid flows in particular. 
Consider a system of linear differential equations in two dimensions described by , 



fdx 

K dt 

K dt 



a u a l2 Tx~ 



(1) 



In general, for N dimensions, \x ] = AX + fi, X — [jc, , X 2 ..x N J ^ 

The two functions x(t) and y(t) represent a curve in the (x,y) plane. The differential 
equation specified in Eqn. (1) specifies how the curve evolves with t. The geometrical 
representation of the qualitative behavior of the curve is known as the phase portrait. 
It can be shown that for a linear system of equations only a finite number of qualita- 
tive phase portraits are possible [6], with the other instances being related by a equiva- 
lence relation. The nature of the flow can be determined through its local properties 
defined by 



curI(X) = (a 2l -a l2 ) 



(3) 



div(X) = trace(A) = (a„ + a 22 ) 
def(X) = J(a u -a 22 ) 2 +(a 2l +a l2 ) 2 
A = yJdef(X) 2 -curl(X) 2 



Table 1. Shows the possible configuration of phase portraits and their required condition. Of 
all the possible configuration, only the spiral and the saddle type of phase portraits are relevant 
to fingerprint images since they correspond directly with loop/whorl and delta type of 
singularities 



Flow Type 


Example 


Condition 


Node 




it 


■ 




\div(X)\>Aand A>0 


■ 


111 


Saddle 




li 


■ 




\div(X)\<Aand A>0 


11 


■ 


Spiral 




ill 




\div(X)\*0and A 2 <0 


Parallel 




111 

HP 




IAI=0 (A is singular) 



(4) 
(5) 

(6) 



3.2 Parameter Estimation 



The orientation image is divided into a system of piecewise linear flows, using a slid- 
ing window centered at each pixel in the orientation image. At each position the cor- 
responding matrix A and B are determined. These represent the parameters of the 
system of linear differential equations that is capable of reproducing the local 
orientation. These parameters can be used to determine the location of the singular 
point as oudined in the next section. 

In order to derive a parameters A,B for each local flow, we adopt a least square mini- 
mization approach that results in a synthesized flow with minimum deviation from the 
observed flow. In particular we try to minimize the distance measures specified by 



X |sin(0-<?) 



(7) 



0(x, y) = arctan — 



= arctan 



a 2[ x + a 22 y + b 2 
a n x + a n y + b x 



(8) 



Here 0(x, y) represents orientation from the synthesized flow and 0(x, y) repres- 
sers the orientation in the observed flow. The distance metric D is invariant to ±180° 
ambiguity between 0(x, y) and 0{x, y) . Other distance metrics such as the acute 

angle between the lines with slopes 0(x, y) and 0(x, y) given by equation (9) are 
also invariant to this phase ambiguity and may be used without affecting the nature of 
the solution. The parameters (all,al2,a21,a22,bl,b2) are evaluated using Levenberg- 
Marquardt non-linear least squares minimization approach. More details of the im- 
plementation is provided in [6]. 

tan 0(x 9 y)-ton0(x,y) (9) 
l + tan0(*,;y)tan£(jt, y) 



The least squares optimization approach renders the estimation fairly robust to noise, a 
common source of errors in the Poincare' index based approaches. The resilience of 
the methods is shown in Figure 3 where the flow is reconstructed in the presence of 
Gaussian noise in the orientation image. 
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Figure 3 (a)Original orientation flow (b) Flow in presence of noise (c) Reconstructed flow 
using linear phase portrait analysis. 

3.3 Position Estimation 

Given the linear phase portrait description, the singularities represent critical points 
where the curve [x(t),y(t)] is stationary. This is specified by the condition, 

[x]=AX + B = 0 (1 

0) 

The position of the singular point is given by, 

x c =-(a-'b) 

However, all types of singularities satisfy this condition. In order to estimate the posi- 
tions of the core and delta positions separately we make use of the classification pro- 
vided in Table 1. As we analyze each section of the orientation image, an estimate of 
the possible location for each singularity is obtained based on Eqn.(9). We use this 
information to update an accumulator that represents the likelihood of each cell being 
a singularity. Separate accumulators are maintained for core(spiral) and delta(saddle) 
singularities. This is similar to the voting approach used in Generalized Hough Trans- 
form. However, unlike Hough Transform, we also increment the vote of the neighbor- 
ing cells in a weighted fashion to make the estimate more robust to noise. 

3.4 Singularity Map Processing 

The resulting position maps I(x,y) can be treated as grey scale images with each pixel 
intensity proportional to the likelihood of the presence of a singularity. In order to 
estimate the position of the singular points the following algorithm is used individu- 
ally on both the spiral and the saddle map resulting from the phase portrait analysis. 

1 . The intensity image is binarized using both a relative and an absolute threshold T! 
and T 2 . Here T x represents threshold on the relative likelihood of the pixel being a 
singular point and T2 represents the absolute threshold indicating the minimum 
vote required by a pixel for it to be considered as a candidate position. The abso- 



(1 
1) 



lute threshold is required to prevent the detection of false singular points in in- 
stances where they are not present in the image at all. This case arises when the 
fingerprint is placed on the sensor such that the core or delta(s) does not appear in 
the image. However, due to noise, some linear neighborhoods may be incorrectly 
classified as candidates for core or delta and may lead to a false detection. The 
absolute threshold helps us to eliminate such spurious candidates. The binary im- 
age M(x,y) is obtained using the rule, 



lif /(jc,;y)>max(7;,:r 2 ) (12) 
0 otherwise 
T x =max{/ (*,>>)}, T 2 =10 



Possible locations where the singularity may lie is obtained by performing con- 
nected component analysis of the thresholded image. Each component is posi- 
tioned roughly centered on each singularity. Outliers that have too few compo- 
nents or too many components are rejected. The resulting map consists of candi- 
date locations each associated with 

The positions of the singular points are determined as the centroids of the candi- 
date locations. For each connected component C 



4 Experimental Evaluation 



We evaluated the algorithm using 100 random images from FVC2002 DB1 database. 
The images were cropped to remove the background prior to the processing. Fig. 3 
and Figure 4 show the results over two such images. 




(a) (b) (c) (d) (e) 



Figure 4 (a)Original Image (b) Orientation Image (c) Spiral map (d) Saddle map (e) 
Results of the spiral/saddle map processing. 



In order to evaluate the accuracy of the proposed method, we estimated the position of 
the singular points in each of the 100 images and compared it with the ground truth 
locations manually marked by the authors. The deviation is measured in terms of the 
absolute distance between the estimated and actual location of the singularities and is 
shown for each sample. Also, the algorithm fails to detect singular points in certain 
cases. A majority of these instances are where the singular points are located at the 
edge of the fingerprint foreground and the absence of the surrounding regions make it 
difficult to detect the deviation in the flow. The summary results are shown in Table 
2. It can be seen that on an average the core and delta position can be localized within 
a ridge width. 
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Figure 5 (a) Regression line for X position of the core location (b) Regression line for Y 
position of the core position. Similar plot can be obtained for the delta location and have been 
omitted for brevity. 



Table 2 Summary results of the algorithm 



Number of core points missed by the algorithm 


8 of 102 


Number of spurious core points 


1 of 102 


Number of delta points missed by the algorithm 


5 of 52 


Number of spurious delta points 


5 of 52 


Mean distance from the actual core (dx,dy) (pixels) 


(9.10,7.24) 


Mean distance from the actual delta (dx,dy) (pixels) 


(5.72,11.32) 



5 Conclusion and Future Work 



In this paper we presented a novel approach for estimation of singular point position 
in fingerprint images using linear phase portraits. The proposed approach presents 
several advantages over existing methods based on Poincare index and in particular its 
robustness to noise and its ability to provide a qualitative description of the flow at 
each point. We also presented an objective evaluation of the algorithm over 100 im- 



ages of FVC2002 DB 1 database and showed that it provides a very accurate estimate 
of the singular point positions. Future work will also involve providing an optimal 
reconstruction algorithm for the orientation image based on the piece-wise linear 
analysis. Currently, the flow parameters evaluated at each location in the orientation 
image leading to an over determined system of piece- wise linear flows. We will ex- 
plore the optimal sampling strategy in order to reduce computational complexity. 
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Curvature Estimation in Oriented Patterns 
Using Curvilinear Models Applied to 
Gradient Vector Fields 

Joost van de Weijer, Lucas J. van Vliet, 
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Abstract— Curved oriented patterns are dominated by high frequencies and 
exhibit zero gradients on ridges and valleys. Existing curvature estimators fail 
here. The characterization of curved oriented patterns based on translation 
invariance lacks an estimation of local curvature and yields a biased curvature- 
dependent confidence measure. Using parameterized curvilinear models we 
measure the amount of local gradient energy along the model gradient as a 
function of model curvature. Minimizing the residual energy yields a closed-form 
solution for the local curvature estimate and the corresponding confidence 
measure. We show that simple curvilinear models are applicable in the analysis of 
a wide variety of curved oriented patterns. 

Index Terms— Oriented patterns, anisotropy, curvature, confidence measures, 
curvilinear models, gradient vector fields. 



1 Introduction 

Reliable estimation of local features in digitized images is of great 
importance for many image processing tasks (segmentation, 
analysis, and classification). Depending on the class of images 
under investigation, knowledge of different features is desired. 
One such class of images is defined by Kass and Witkin [1] as 
oriented patterns: patterns that exhibit a dominant local orienta- 
tion. Examples are seismic, acoustic, wood grain, interference 
patterns, and fingerprint images. Important features for these 
images are estimates of local anisotropy, orientation, curvature, 
and scale. 

The structure tensor yields a robust estimator for local 
orientation [1], [2], [3], [4] based on a local gradient vector field. 
This estimator locally models the images as translation invariant 
strokes. In addition to orientation estimation, this method also 
yields an anisotropy measure indicating the resemblance of the 
local area to a translation invariant model. This measure can also 
be interpreted as a confidence measure of the estimated orienta- 
tion. Both a lack of smoothness (e.g., caused by noise) and 
deviations from the translation invariant model (e.g., curved 
oriented patterns) are responsible for a decrease of this confidence 
measure. To distinguish between the two possible causes, we 
proposed a parabolic transformation which optimizes the transla- 
tion invariance after transformation [5]. This method yields a 
curvature estimate for curved oriented patterns as a by-product. A 
shortcoming of this method is that the proposed transformation is 
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not orthonormal and, therefore, it lacks conservation of gradient 
energy. This does not allow direct comparison of the confidence 
values of different transformations. In this paper, we propose a 
method to investigate the resemblance of a local partem of 
2D oriented pattern to a certain model function (e.g., circular, 
parabolic). The model is represented by a parameterized transfor- 
mation function of the isophotes. The method assures the 
conservation of gradient energy, allowing us to compare con- 
fidence measures of different transformations and, especially, of a 
parameterized transformation for different parameter values. As in 
[5], solving the parameter for best confidence yields a closed-form 
estimate of the additional free parameter, e.g., local curvature. We 
propose two curvilinear models, a parabolic and a circular model, 
for the characterization of curved oriented patterns. When the 
resemblance between a model and a local image is high, the 
corresponding model parameters, orientation, and curvature yield 
a reliable description of the local image. The method yields 
features with a corresponding confidence value. All these 
estimates are local and can be represented as feature maps. 

Estimation of the curvature in oriented patterns is not trivial. 
Worring and Smeulders [6] presented an extensive comparison 
between curvature estimators applied to segmented data for which 
the position and ordering of points along the contour have to be 
known. For noisy oriented patterns, segmentation fails, making 
these methods useless. The isophote (tangential) curvature (the 
second derivative along the isophote divided by the gradient 
magnitude) and the normal curvature [17] are segmentation-free 
[7], [8], [17], but also fail on these images. There are three reasons 
for this [5]: 1) The gradient is zero on ridges and in valleys. 
2) Increasing the regularization scale of directional derivatives 
suppresses the oriented pattern and reduces the signal-to-noise 
ratio. 3) Opposite sides of a ridge (or valley) yield curvatures of 
opposite sign which cancel out after averaging. The only two 
methods which do yield a curvature estimate for oriented patterns 
are either very computationally demanding [9] or are not 
accompanied by a confidence measure, which makes them hard 
to rely on [10]. 

The proposed method resembles a method for the detection of 
complex symmetries as presented by Bigun et al. [11], [12], [13]. 
They characterize symmetries by (coordinate) transformation 
functions which transform symmetric patterns into translation 
invariant patterns. The success of such a transformation is 
determined by the confidence measure of the structure tensor 
applied to the transformed image. A high confidence value is an 
indicator for the presence of the corresponding symmetry. Bigiin 
et al. method is an extension of the generalized Hough transform. 
Detection of a symmetry pattern involves accumulation of 
evidence by voting. Bigiin's symmetry detector requires two 
orthonormal transformation functions. It measures the resem- 
blance of the local differential field to two perpendicular 
differential fields (indicating the symmetry), whereas our method 
looks at the resemblance of the local differential field to only one 
differential field (representing the shape of the isophotes). This 
difference allows us to estimate model parameters by optimizing 
the resemblance between the actual differential field and a model 
differential field in a closed-form solution, i.e., omitting a time, 
consuming voting scheme. This is not possible with the symmetry 
method since neither one of the two differential fields is preferred. 
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The requirement for two orthonormal transformation functions 
poses an unnecessary limitation to the symmetries. For example, 
such a set of functions does not exist for the parabolic model we 
propose, i.e., parabolic isophotes along a linearly increasing 
symmetry axis. We extend his method by noting that only the 
existence of the differential fields of the two transformation 
functions is essential. 

2 Oriented Patterns 

An oriented pattern m(x, y) can be written as a real one 
dimensional function g of a model function u 



™>(x,y) = g{u(x,y,a)). 



(1) 



The model function u(x, y, a) describes the shape of the isophotes 
and a contains local isophote parameters such as orientation and 
curvature. Consequently, the gradient (differential field) of m, 



da 

Vm = -f- Vu, 

du 



(2) 



is a dg/du weighted version of the gradient of u. In oriented 
patterns, we distinguish between two perpendicular orientations; 
along the isophote (tangent) and along the gradient. Note that 
orientation is defined on the interval [0,7r). Consequently, vectors 
in opposite directions have the same orientation. 

Consider the function f(x,y) representing a local image 
(window) and a model function u(x, y> a). It is of interest to what 
extent f(x y y) is described by the model function u(x,y,a). This is 
measured by decomposing the derivative energy of /(x, y) into two 
contributions, one parallel and one perpendicular to the 
normalized differential field of u(x,y,a). This results in the 
following energies: 



*«-//("m)** 



(3) 



where E f (a) denotes the fit energy and E r (a) the residual energy. 
The subscript J_ indicates a rotation of 90° of the vector and the 
integrals represent the averaging over the local image. A quality 
measure of the fit can be found by comparing the fit energy with 
the residual energy. Since no a priori knowledge exists to interpret 
the energy difference between the fit and the residual energy, we 
normalize the difference with the total gradient energy to obtain 
the following quality measure c(a): 



c(a) - E/(»)-£r(a) 
° W £; / (a) + £ r (a) 



- 1 < c < 1. 



(4) 



The value of c(a) varies from -1 for a pattern of which the 
isophotes are exactly perpendicular to those of the model function 
u(x t y, a) and +1 for a pattern which is exactly described by the 
model function. The isotropic noise energy is distributed equally 
between the fit and the residual energy. 

More important than the quality measure for an arbitrary a is to 
know which a maximizes the quality function c, i.e., maximizes Ej 
and minimizes E T . The vector a contains model parameters which 
describe local features. Therefore, optimizing the confidence 
function c corresponds to feature estimation. Furthermore, the 



quality measure c(a) informs us about the success of the fit and can 
be seen as a confidence measure of the estimated features. Besides 
comparing confidence measures of the same model function, it is 
also possible to compare confidence measures of different model 
functions. Note that the normalization of the confidence measures 
is independent of the model function. By comparing optimized 
confidence functions of various models, one can find out which 
model describes the local pattern best. 

Usually, the complexity of the confidence function does not 
allow a closed-form solution of the optimization criterion. The 
straight model is an exception. In the case of curvilinear models, 
we avoid costly (iterative) optimization procedures by considering 
approximate confidence functions which do allow closed-form 
solutions. 

3 Straight-Oriented Patterns 

Locally, many oriented patterns can be characterised by a straight 
model. For such a pattern the model function u(x, y, a) is given by 

u{x,y,4>) = x cos <j> + t/sin 0, (5) 

with <j> the orientation perpendicular to the model isophotes. 
Substituting this in (3) yields 

^/W = 5(^ + ^)+5(l-^)cos20 + i 2££sin20. (6) 

A bar (•) denotes an averaged quantity and will from now on 
replace the integrals responsible for averaging over a local image. 
The confidence value c{<j>) is 

C(0) = JITJI ((£ " Tfj cos ^ + VJv*™ (7) 

c(<f>) can be maximized as a function of the orientation <f>. This 
yields the following (gradient-based) orientation estimator [1], [2], 
PL [4]: 



1 . Vxfy 

i = -arctan — 

2 si-n 



(8) 



with confidence value c($opt)' 

= > * = TTlf + Vjy. (9) 

This confidence measure can also be interpreted as a measure for 
translation invariance and shows an intuitive dependency to the 
pattern orientation 



c{4>) = 



_ <P(cos 2 (<j> - Qwt) - sin 2 (0 - 



9 s 



= i c(^)(l + cos(2(^-^))). 



(10) 



The maximum of the confidence measure c(^) reduces due to 
noise in the local image /. For a linear pattern p distorted by 
additive uncorrected noise n (/ = p + n), the confidence value 
c^^) is: 



C(<Popt 



) = 



l|Vp|| 2 



IIV/H 2 ||Vp + Vn|| 2 ||Vp|| 2 ||Vp|| J + ||Vn|| s 



(11) 
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Fig. 1 . Confidence measure c(o, k) of circular, parabolic, and straight line models 
on a noise-free pattern of concentric circles. 

Note that the gradient noise energy is divided equally over Ej and 
E r . Therefore, the numerator of c is unaffected by noise. Noise 
increases the total gradient energy (denominator of c), which 
lowers the confidence value c^f). Another reason for a lower 
confidence value is a deviation between the local image and the 
model function. For instance when curved lines occur, then 
curvature will contribute to E r . In the next section, we will extend 
the model to include curved patterns. 

4 Curved Oriented Patterns 

We present two model functions which locally model curved 
oriented patterns. A parabolic model 



u(x, y, <f>, k) 
and a concentric circle model 



- - KV? — V 



u(x,y,<j>,K) = \J k 2 w 2 + (1 — kv) 2 



(12) 



(13) 



in which n is the curvature. The Gauge coordinates v, w are 
obtained by 



v — x cos <f> + y sin <f> 



w = -x sin <f> + y cos 0 



(14) 



Here, we discuss the parabolic approximation. For the circular 
approximation, we refer to Appendix A. Using the parabolic 
model function and (3), the following energies are obtained: 



E f^^={ TT^P > 



' K*W*fi + 2KWf v f w + p w 
1 + K 2 W 2 



(15) 



where /„ and /„, are the derivatives in, respectively, the v and 
w direction. Finding the curvature and orientation that maximize 
the confidence function requires a search in 0, /c-space. In this 
paper, we shall not further investigate this method due to its high 
computational demands. Instead, we propose a way to approx- 
imate the confidence function, allowing a fast closed-form solution. 

An approximation to the orientation <p can be obtained by 
looking at the axis of minimal translation invariance for parabolic 
and circular patterns. In the case of a circular pattern, this is the 
t^axis. For a parabolic pattern, it depends on the curvature and the 
window size used. For small curvatures (i.e., compared to the 
window size), the minimal translation invariant axis is equal to the 
uf-axis. Increasing the curvature, the axis of minimal translation 
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Fig. 2. (a) Average confidence measure c(0, k) for the circular model as a function 
of the radius for three different SNRs (top to bottom: 20 dB, 10 dB, 6 dB). The 
measure c($, k) yields a small bias for small radii. The horizontal lines indicate the 
average confidence measure for, straight model for the corresponding SNR. 
(b) Probability density functions of the confidence measures for the straight- 
oriented patterns for the three different SNRs (top to bottom: 20 dB, 10 dB, 6 dB). 

invariance jumps to the tu-axis. Therefore, an approximation of the 
orientation needed to deterrnine the v andu^axes in (15) can be 
computed with (8). After substituting the orientation, the resulting 
equations only depend on the curvature. Iterative maximization of 
the confidence function in K-space is still time-consuming. We 
propose approximating this maximum by using locally adapted 
weighting. The weighting function of Ej and E r (denoted by the 
bar •) is in its turn weighted by (1 + k 2 vP) after which we 
normalize for this weighting. This mathematical trick has a high 
resemblance to normalized convolution [14]. It results in 



1 + K 2 W 2 



E r (K) = 



k 2 ^! 2 + 2nwf w f v + fl 



(16) 



1 + kV 

A hat (•) above a quantity indicates an approximation. Since the fit 
energy Ej might be a function of the coordinate w, as is the 
adapted weighting function, optimization lead to a false curvature 
estimate. Therefore, minimization of the residual energy E T is used 
to find the following closed-form curvature estimate: 



(17) 



2w 2 • wf w f v 

The confidence measure can now be computed in two different 
ways. The confidence measure c(4>, k) has its maximum at ^ , /c^. 
To avoid an interative search for this optimum, one can compute 
c(4>, k) by subsitiuting <f> and k in (15) and (4). Note that estimates 4> 
k do not have to be equal to the values of <j> and k that optimize the 
confidence function. However, computing c(<£, k) is still expensive. 
A significant speed-up can be obtained by approximating the 
confidence measure using the approximate energies of (16). 
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c(4>, k) = 



E f (<t>,K)-E r (<t>, k) 



(18) 



100 



Again, one can avoid an iterative search by substituting <f> and k in 
(18), which yield c(0, k). 

The curvature estimator in (17) is the tangential or isophote 
curvature. The normal (or gradient flow line) curvature [17] can be 
computed by exchanging the v and w coordinates in (12) and (13). 

5 Implementation 

Direct computation of the curvature and the confidence measure is 
a space-variant operation. This yields a high computational 
demand. Fortunately, (16) to (18) can be calculated with global 
convolutions which can be implemented by multiplication in the 
Fourier-domain. This yields a substantial reduction in computa- 
tional complexity. The derivatives f x and f y are implemented as 
regularized derivative filters: 

dg\X) y\ Cg) f 

fx = f(x,y) & dx « — ► jwj(uj £ ,bj y )g(u x ,u} y ;<j g ), (19) 

with / the Fourier transform of / and g(x,y;a g ) a Gaussian 
regularization function of scale a g . 

9(x,V,<r s ) = 2^J e " (l2V)/ ^ ~ = e-K^~!K. (20) 

The terms of the curvature estimator and the confidence measure, 
(16) and (17) , are expanded in Appendix B (the circular model is 
treated in Appendix A). The remaining terms zP&fZf* are 
conveniently calculated as multiplications in the Fourier domain 

^JfltJi = 7, <Ta) ® fjy ~ «(P, <?, <r a )F{r x r y }. (21) 

For the window function, we choose a Gaussian of scale a a - The 
Fourier transform of the filter uQ is 

u(p, g, a.) = x"y"g(x, y; c a ) ^ u(p, = f*< ° m) ■ 

(22) 

Due to the high frequency character of oriented patterns, a g should 
be kept small, i.e., tuned to the frequency characteristics of the 
cross-section of a line. Noise suppression is accomplished by 
averaging all terms by Gaussian window (size <r a ), i.e., the size of 
the curvilinear model. 



6 Experiments 

In this section the proposed algorithms are tested on synthetic and 
real-world images. The feature extraction which we presented is 
based upon finding a maximum of the confidence measure in 
parameter space c(a). The curvature of oriented patterns corre- 
sponds to the position of the maximum in c(k, <£)-space. To avoid 
searching /t,0-space, the approximations 0 and k are proposed. 
With these, an approximated confidence measure c and the exact 
confidence measure c may be computed. The goal of the 
experiments is to investigate the performance of these approxima- 
tions as a function of the curvature. Also, the robustness with 
respect to the noise is checked. The tests are performed on a 
concentric circle image f(x, y) = sin^x 2 + y 2 + <pj + n (see Fig. 1) 
in which n = N(0,al) and (p is a phase-term set randomly for 
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Fig. 3. Rms error between the actual confidence measure and its approximation as 
a function of the radius, (a) Approximate parabolic model applied to concentric 
circle patterns of various SNR (top-to-bottom: 6 dB, 1 0 dB, 20 dB). (b) Approximate 
circular model applied to concentric circle patterns of various SNR (top-to-bottom: 
6dB, 10 dB, 20 dB). 

every noise-realization. For the signal-to-noise ratio, we use 
SNR = 10 log(/i 2 /^), where h is the contrast difference and a n 
the standard deviation of the noise. Be aware that the proposed 
algorithms are based on the gradient energy of the local image. 
Thus, an increase of the pattern frequency will usually result in a 
higher SNR (gradient energy versus filtered noise variance) and, 
therefore, a better performance. All experiments on the concentric 
circle image are based on 100 measurements. Unless mentioned 
otherwise the sigma sizes are a g = 1.0 and a a = 5.0. 

Confidence measure as selection criterion. The importance of 
choosing the right model is illustrated in Fig. 1, which shows the 
confidence measures of the circular, parabolic, and straight model 
applied to a noise-free pattern of concentric circles. It is clear that, 
for high curvatures, the deviation of the straight and the parabolic 
model form the circle pattern results in a significantly lower value 
of the confidence measure. 

Bias of the actual confidence measure. To investigate to what 
extent the optimum of the confidence function in k, <£-space is 
found, we compare the average confidence measure of the circular 
model applied to curved patterns with the average confidence 
measure of a straight model applied to a straight pattern. Both 
images have identical signal-to-noise ratios. The confidence 
measure c(<f>, k) of a curvilinear model can be slightly higher than 
the confidence measure of a straight model. This slight increase is 
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Fig. 4. Curvature estimators using curvilinear models: black line = parabolic model, gray line = circular model, (a), (b) and (c) Coefficient-of -variation (CV) for the parabolic 
and circular model-based curvature estimators for different SNR (20 dB,10 dB, 6dB). (d) Bias of parabolic and circular model based curvature estimators (SNR = 10 dB) 
(thick gray line indicates the noise-free bias of curvature using the parabolic model). 




caused by the fact that the curved model allows for two parameters 
to adjust to the noise. 

The average confidence measure c(4>, k) of the circular model 
applied to the concentric circles is depicted in Fig. 2 for three SNR's 
(20dB, lOdB, 6dB). It clearly shows that, for small radii, the average 
confidence measure c(<j>,k), decreases. This is caused by an 
increasing discrepancy between the approximated (0, k) and the 
optimal (<j>opuKopt) for small radii. Note, c(^ f ,K op( ) does not 
decrease for small radii. Fig. 2b indicates the variation around the 
average confidence measure for the straight model. Increasing the 
window size (local image) reduces the variation in exchange of a 
further decrease of c(0, k) for small radii. 

Approximation error of the confidence measure. In Section 4, 
we presented two methods for computing the confidence measure, 
the actual confidence measure c(4>,k) and an approximation 
c(<j>, k). In Fig. 3, the rms (root-mean-square) error due to this 
approximation is depicted for the circular and the parabolic model. 
For both models, these errors are small. Only for high curvatures 
(small radii), it may be worthwhile to compute the actual 
confidence measure. 

Robustness of the curvature estimator. It is important to test 
the robustness of the curvature estimation. In Fig. 4, the noise 
sensitivity of the parabolic and circular curvature estimators is 
depicted. Both models were applied to the concentric circles. The 
coefficient-of-variation (CV = o/ii) of both models is similar for 
the middle and high SNRs, but the parabolic models performs 
better for low SNRs. Considering the advantage of the circular 
curvature estimator due to the exact match between the model and 
the pattern, we show that the parabolic curvature estimator suffers 



less from the approximations. The parabolic curvature estimator 
performs at least as well over a wide range of curvatures. Only for 
high curvatures, the circular model can take advantage of the exact 
match. In practice, one can compute the curvature corresponding 
to both models. The one with the highest confidence measure is 
preferred because its model yields a better description of the data. 

Application of curvilinear models to real-world data sets. In 
Fig. 5, an interference pattern, together with the curvature and 
confidence estimation for both the parabolic and circular model, is 
depicted. As expected, the parabolic model fails in the middle of 
the ellipses, as indicated by an abrupt drop of the confidence 
measure. The circular confidence measure hardly decreases for the 
circles at the top and the bottom of the image. For the flatter 
ellipses on the left and the right, the mismatch between the model 
and the pattern is slightly larger. In the difference image between 
the circular and parabolic confidence measures, the lighter areas 
indicate a better description of the circular model, whereas, in the 
darker areas, the parabolic model yields a better fit. The slightly 
darker lines denote an almost perfect parabolic line pattern. The 
isophote curvature fails at the ridges (dark lines) and valleys 
(white lines) as expected (see Section 1). 

The estimated local curvature of a fingerprint and a CT cross- 
section of a tree-trunk are depicted in Fig. 6. Both curvilinear 
models produced similar results. The dark lines in the logarith- 
mically stretched curvature images denote locally straight patterns. 
Both peaks in the fingerprints curvature correspond to important 
minutia for fingerprint recognition [15], [16]. The isophote 
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Fig. 5. (a) Interference pattern of a vibrating plate. The superimposed circle denotes the size of the curvilinear model, (b) and (c) Confidence measures for, respectively, 
the circular and parabolic model (range [0,1]) computed with a g = 1.0 and <j a = 5.0. (d) Difference in confidence measure between circular and parabolic model (range 
[-0.5, 0.5]). (e) and (f) Estimated curvatures k for respectively the circular and parabolic model (log stretched), (g) Isophote curvature at scale a - 5 (range [-1,1]), 



curvature fails at the ridges (dark lines) and valleys (white lines) as 
expected (see Section 1). The curvilinear curvature can be used to 
improve (to prevent jumping the rails) the ridge tracking [16], 



which is already based on orientation estimation. The high 
confidence measures (white areas in confidence images) indicate 
a perfect fit of the model and a reliable curvature estimate. 
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Fig. 6. (a) Fingerprint image, (b) CT image of trunk, (c) and (d) The estimated curvature k using the parabolic model (log stretched) at a, = 1.0 and <r 0 = 5.0. (e) and 
(f) The confidence measure of the parabolic model (range [0, 1)), (g) and (h) Isophote curvature at scale a = 5 (range [-1, 1]). 

7 CONCLUSIONS is obtained by optimization of the quality function as a function of 

In this paper, we present a method to compare a local image with a the parameters which represent the feature. The quality function is 
model function. A quality measure indicates the resemblance interpreted as a confidence measure for the measured features. We 
between the local image and the model function. Feature extraction propose two curvilinear models to describe curved oriented 
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patterns. To avoid searching <£, K-space, we propose, closed-form 
solution for approximations to the actual parameters of the 
curvilinear models 0 and k. Instead of the exact confidence 
measure c(<£,k) an approximation c(<£, k), can be computed 
resulting in a huge reduction in computational demand. We 
demonstrate that these approximations yield good results for 
almost all curvatures. Only for the highest curvatures, one might 
decide to compute c(<£, k) or (even more computationally demand- 
ing) to iterate in <j>, K-space for c(<f>opt, «opt). 

Appendix A 

For a concentric circle model, u(x, </>, k) = \J k 2 w* + (1 - kv) 2 , the 
fit and residual energies are 



' 1 (1-/™) 2 + kV 



p (k) = Al - Kvffi, + 2HW{1 - KV)/J W + K' 

\ (l-™f+tV 



(23) 



To obtain a closed-form solution for the curvature and the 
confidence measure, the local energies are computed inside a 
{k 2 v? + (1 - Kv) 2 ^j -weighted space-variant window. This yields 

~ 2 {ff 2 + 2vwf v f w + yfif*) + 2k{-vP v - wf v f w ) + J 2 



=7 =• 



1 -2kv + k 2 (v* + u> 2 ) 



Ak 2 + 2Bk + C 



_ (24 ) 

k 2 {v 2 P w - 2vwf v f w + w 2 ft) + 2k{-vP w + wf v f w ) + f 2 



1 - 2kv + /c 2 



£/t 2 + 2F/c + G 
X + Dk 2 ' 



with v = 0. The minimization of the residual energy yields an 
approximation of the curvature: 



_E-GD- yj AF 2 D + (-E + GD) 2 
* = 2FD * 

The terms of Ej andE g are expanded with (14) and 

f v = fx cos (p + f y sin 0 / u , = ~f x sin 0 -f / y cos 0. 
This results in 



(25) 



(26) 



A = x 2 p^ + 2xyf x f y + y>% ^ 
B=-(xfi+^J^cos<l>- (xjj y+ ^ sin0 
C = fl cos 2 <j> + cos 0 sin 0 + sin 2 0 

£>=M_ (27) 

E = x 2 p y -2xyf x f y + y>p x ^ 

F = (^/-Jy ~ x/ 2 ) cos0 4- (xf~T y - tff) sin0 
, G = cos 2 0 - 2/b/p cos 0 sin 0 + sin 2 0. 

The averaged terms can be computed as global convolutions (see 
Section 5). The approximated confidence function is computed with 



k 2 {A - E) + 2k(B - F) + (C - G) 
« 2 (yl + £) + 2k(5 + F) + (C + G) ' 



(28) 



Appendix B 

The terms for the parabolic confidence measure (16 ) and curvature 
estimator (17 ) are 

' = cos 4 0 - 2^f 2 + JJJ y ) cos 3 0sin0 
+ {ff 2 + + ^ ) cos 2 0sin 2 0 

+2 (-x 2 / x / y - iy/f) cos 0 sin 3 0 -f i^sin 4 0 
w 2 / 2 = V 2 Jl cos 4 0 - 2(^f 2 - y 2 f x f y > j cos 3 0sin0 
+ [x 2 fl ~ teyfxfy + y 2 ^) cos 2 0sin 2 0 (29) 
+2(x 2 f x f y - 2xy^ cos0sin 3 0 + S^sin 4 0 

™/tJu = J//i/ v cos 3 0+ (^-xf x f y -y^ft-fy 1 )^ cos 2 0sin 0 

+ (* (/2 - /J) - 5£7») cos 0 sin 2 0 + £££ sin 3 0 
^w 2 = ol 

for /J and /J, see term, C and G in Appendix A. 
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Normal Curvature 

Let u p be a unit tangent vector of a regular surface M C ffi^ . Then the normal curvature of 
M in the direction u p is 



k(u p ) =S(u p )u p) 



(1) 



where S is the shape operator. Let Af C ffi 3 be a regular surface, P € M r x be an injective 
regular patch of M with P = x(uot v o) , and 



where v p € M p , Then the normal curvature in the direction v p is 



(2) 



K(vp) = 



ea 2 +2fab + gb 2 
Ea 2 +2Fab+G&* 



(3) 



where E, F, and G are the coefficients of the first fundamental form and e, f t and g are the 
coefficients of the second fundamental form. 

The maximum and minimum values of the normal curvature at a point on a regular surface are 
called the principal curvatures K i and ^2 . 
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Shape Operator 



The negative derivative 



S(v) = -D V N 



(l) 



of the unit normal N vector field of a surface is called the shape operator (or Weingarten map 
or second fundamental tensor). The shape operator S is an extrinsic curvature, and the 

Gaussian curvature is given by the determinant of S. If X : 17 -4 IR^ is a regular patch, then 



S(x v ) 



(2) 
(3) 



At each point p on a regular surface M C / the shape operator is a linear map 

S : M p -+ Af p . (4) 

The shape operator for a surface is given by the Weingarten equations. 
j|| AlSO: Curvature, Fundamental Forms, Weingarten Equations 
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Abstract 

We describe the design and implementation of an on- 
line fingerprint verification system which operates in two 
stages; (i) minutia extraction and (ii) minutia matching. An 
improved minutia extraction algorithm that is much faster 
and more accurate than our earlier algorithm [8] has been 
implemented. For minutia matching, an alignment-based 
elastic matching algorithm has been developed. This algo- 
rithm is capable of finding the correspondences between in- 
put minutiae and the stored template without resorting to 
exhaustive search and has the ability to adaptively com- 
pensate for the nonlinear deformations and inexact pose 
transformations between fingerprints. The system has been 
tested on two sets of fingerprint images captured with ink- 
less scanners. The verification accuracy is found to be over 
99% with a 15% reject rate. Typically, a complete finger- 
print verification procedure takes, on an average, about 8 
seconds on a SPARC 20 workstation. It meets the response 
time requirements of on-line verification with high accu- 
racy. 



a variety of personal identification and access control appli- 
cations. The overall block diagram of our system is shown 
in Figure 1. It operates as follows: (i) off-line phase: a dig- 
ital image of one fingerprint of a person to be verified is 
captured; a feature extraction algorithm is applied; minu- 
tiae are extracted and stored as a template for later use; (ii) 
on-line phase: the individual to be verified indicates his/her 
identity and places his/her finger on the inkless scanner; an 
digital image of his/her fingerprint is captured; minutiae are 
extracted from the captured image and fed to a matching 
algorithm, which matches it against the stored template. 

Off-line processing 
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Figure 1. Overview of the system 



1. Introduction 

Fingerprint verification is one of the most reliable per- 
sonal identification methods [2, 4]. It plays a very impor- 
tant role in forensic and civilian applications such as crimi- 
nal identification, access control, and ATM card verification 
[4]. However, manual fingerprint verification is so tedious, 
time-consuming, and expensive that it is incapable of meet- 
ing today's increasing performance requirements. As a re- 
sult, automatic fingerprint identification systems (AFIS) are 
in great demand [4], 

In this paper, we will introduce an on-line fingerprint 
verification system which is capable of verifying identities 
of people in "real time". Such a system has great utility in 



In the following sections, we will describe in detail 
our on-line fingerprint verification system. Section 2 
mainly discusses the feature extraction algorithm. Section 3 
presents our minutia matching algorithm. Experimental re- 
sults on fingerprint databases captured with inkless scanners 
are described in section 4. Section 5 contains the summary 
and discussion. 

2. Minutia Extraction 

We have implemented a minutia detection algorithm 
which is a modified version of the technique proposed by 
Ratha et al. [8]. The overall flowchart is depicted in Fig- 
ure 2. 
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Figure 2. Flowchart of the minutia extraction 
algorithm 



2.1. Estimation of Orientation Field 

A new hierarchical implementation of Rao's algorithm 
[7] is used to estimate the orientation field of an input fin- 
gerprint image. It consists of the following two main steps: 

1 . Estimate the local orientation at each pixel using fol- 
lowing formula: 



1 Et=i £j=i (<?;(*» j) - 

where W is the size of the local window; G x and G y 
are the gradient magnitudes in x and y directions, re- 
spectively. 

2. Compute the variance of the orientation field in a lo- 
cal neighborhood at each pixel (z, j). If it is above a 
certain threshold T Vi then the local orientation at this 
pixel is re-estimated at a lower resolution level until it 
is above the threshold value. 

Experimental results show that even in the presence of 
noise, smudges, and breaks in ridges, a fairly smooth ori- 
entation field estimate can be obtained with this algorithm. 
Before the ridge detection, a segmentation algorithm which 
bases its judgment on the local variance of grey levels is 
used to locate the fingerprint region in the image. 

2.2. Ridge Detection 

The most salient property corresponding to ridges in a 
fingerprint image is that grey level values on ridges at- 
tain their local maxima along the normal directions of lo- 
cal ridges. In our minutia detection algorithm, a finger- 
print image is first convolved with the following two masks, 
h t (x,y\u,v) and hb{x,y\ u,v) of size W x H, which are 
capable of adaptively accentuating the local maximum grey 
level values along the normal direction of the local ridge 



directions: 



h t {x,y\u,v) 



e* 2 



I 0, 



( -1 



h b {x,y\u,v) = < 



l(v) 
d 



\iu = i(v) - d, v e n 
ifu = j(v),uen ( 2 ) 

otherwise, 
e^r, if tx = l(v) + G n 
7=^^, ifu = l(v) t v € H ( 3 ) 
, 0, otherwise, 
vtan(0(x,y)), (4) 
H 

(5) 



n = h 



2cos(0(x,y))' 
sm(Q{x,y)) 



-2 



sin(^(x,y)) 



, (6) 



where 0(x, y) represents the local ridge orientation at pixel 
(x,y). If both the grey level values at pixel (x,y) of the 
convolved images are larger than a certain threshold T ry 
then pixel (x,y) is labeled as a ridge. By adapting the 
mask width to the width of the local ridge, this algorithm 
can efficiently locate the ridges in a fingerprint image. Af- 
ter the above steps, a thinning algorithm is applied to obtain 
a thinned 8-connected ridge map. 

2.3. Minutia Detection 

Without a loss of generality, we can assume that if a 
pixel is on a thinned ridge (8-connected), then it has a 
value 1, and 0 otherwise. Let N 0 ,Ni,...,N 7 denote the 
8 neighbors of a given pixel (x,t/), then pixel (x,y) is a 
ridge ending if Y^i=o = 1 and a ridge bifurcation if 
£f=o > 2. However, the presence of undesired spikes 
and breaks present in a thinned ridge map may lead to many 
spurious minutiae being detected. Therefore, the following 
heuristics are used in preprocessing: (i) If a branch in a 
ridge map is orthogonal to the local ridge directions and its 
length is less than a specified threshold T bi then it will be 
removed; (ii) If a break in a ridge is short enough and no 
other ridges pass through it, then it will be connected. 

For each minutia, the following parameters are recorded: 
(i) x-coordinate, (ii) y-coordinate, (Hi) orientation, and (iv) 
the ridge on which it resides. The recorded ridges are nor- 
malized by average local ridge distance along the normal 
direction of local ridges. Figure 3 shows the results of our 
minutia extraction algorithm on a fingerprint image cap- 
tured with an inkless scanner. 

3. Minutia Matching 

Generally, an automatic fingerprint verification is 
achieved with minutia matching (point pattern match- 
ing) instead of a pixel-wise matching or a ridge pattern 




(a) input image 



(b) orientation field 




(d) ridge map 



(f) extracted minutiae 



Figure 3. Results of the minutia extraction al- 
gorithm on a fingerprint image (640 x 480) cap- 
tured with an inkless scanner. 



matching of fingerprint images. Because a general point 
matching problem is essentially intractable, features as- 
sociated with each point and their relative positions are 
widely used in the point pattern matching algorithms to 
reduce the exponential number of search paths [3, 1, 6, 9]. 
However, these algorithms are inherently slow and are 
unsuitable for an on-line fingerprint verification system. 
In our system, an alignment-based matching algorithm is 
implemented, which decomposes the minutia matching 
into two stages: (i) alignment stage, where transforma- 
tions such as translation, rotation and scaling between 
an input pattern and a template are first estimated; the 
input patterns are then aligned with the template ac- 
cording to the estimated parameters; and (ii) matching 
stage, where both the input pattern and the template are 
converted to polygons in polar space and an elastic string 
matching algorithm is used to match the resulting polygons. 

Let P = ((xf , y P , 0P)T, .... (fl £ , y P 9 QPf) and Q = 
((x?,y? J 0f) T ,...,(x% ) y% } 0%) T ) denote the M minu- 
tiae in the template and the N minutiae in the input image, 
respectively. The steps of our alignment-based matching al- 
gorithm are given below: 

1. Match the ridge associated with each input minutia 
against the ridge associated with each template minu- 
tia and align the two patterns according to the matching 



result. 

2. Convert the representations of template and input 
minutiae into the polar coordinate representations with 
respect to the corresponding minutia on which align- 
ment is performed and represent them as two symbolic 
strings by concatenating each minutia in an increasing 
order of radial angles: 



P\T 



J ( r M» e M^M) T ) 



(7) 
(8) 



where r + , e*, and 6* represent the corresponding ra- 
dius, radial angle, and normalized minutia orientation 
with respect to the reference minutia, respectively. 

3. Match the resulting strings P p and Q p with a modified 
dynamic-programming algorithm described below to 
find the 'edit distance' between P p and Q p . 

4. Compute the matching score of the template and input 
minutiae as the minimum 'edit distance*. 



M _ lQQivw 

pq ~ max{M,N}' 



(9) 



where N pa i r is the number of minutia pairs which fall 
within a given bounding box. 

The maximum and minimum values of the matching score 
are 100 and 1, respectively. The former value indicates a 
perfect match, while the later value indicates no match at 
all. 

3.1. Alignment of Point Patterns 

It is well known that corresponding curve segments are 
capable of aligning two point patterns with high accuracy 
in the presence of noise and deformations. Each minutia in 
a fingerprint is associated with a ridge. A true alignment 
can be achieved by matching and aligning the correspond- 
ing ridges (see Figure 4). By matching the corresponding 
normalized ridges, the relative pose transformation between 
the input minutiae and the template can be estimated. With 
the estimated pose transformation, the input minutiae can 
then be translated and rotated to align the template minu- 
tiae. 

3.2. Aligned Point Pattern Matching 

If two identical point patterns are exactly aligned, each 
pair of corresponding points are completely overlapping. 
In such a case, a point pattern matching can be simply 
achieved by counting the number of overlapping pairs. 
However, in practice, such a situation is rarely encountered. 




matching process. It can be represented as follows: 



Figure 4. Alignment of the input ridge and the 
template ridge. 



£l(m,n) 
Template minutia 




Input minutia 



- Reference minutia 



Figure 5. Bounding box and its adjustment. 



The error in determining and localizing minutia hinders the 
alignment algorithm to recover the relative pose transforma- 
tion exactly. Also, in the presence of nonlinear deformation 
of fingerprints which is an inherent nature of fingerprint im- 
pressions, it is impossible to exactly recover the position of 
each minutia with respect to its corresponding minutia in 
the template. Therefore, the aligned point pattern matching 
algorithm needs to be able to tolerate, to some extent, the 
deformations due to inexact extraction of minutia positions 
and nonlinear deformations. 

The symbolic string generated by concatenating points 
in an increasing order of radial angle uniquely represents a 
point pattern. A modified string matching algorithm which 
incorporates an elastic term is capable of achieving a certain 
type of tolerance. However, this algorithm can only tolerate, 
but not compensate, the adverse effect on the matching pro- 
duced by the inexact localization of minutia and nonlinear 
deformations. Therefore, an adaptive mechanism is needed, 
which should be able to track the local nonlinear deforma- 
tions and inexact alignment and try to alleviate them dur- 
ing the minimization process. In our string matching algo- 
rithm, this adaptation is achieved by adjusting the bounding 
box (Figure 5) when an inexact match is found during the 



£i(m+ l,n + 1) 
<^(m+ l,n + 1) 
ei(m + l,n + 1) 
€h(m + l,n + 1) 



8i(m,n) +r?Ar a , (10) 

Sh(m,n) + 7/Ar a , (11) 

€i(m,n) + 7?Ae a , (12) 

€/ 1 (m ) n) + fjAe ai (13) 



where £/(m,n), 6k(m,n) 9 €/(m,n), and e^m^n) specify 
the adaptive bounding box in radius and radial angle; Ar a 
and Ae 0 represent the current difference of input and tem- 
plate in radius and radial angle, respectively; rj is the learn- 
ing rate. In our system, the initial values of the bounding 
box are specified as follows: ^(0, 0) = -8; ^(0, 0) = +8; 
e«(0> 0) = -4; Cfc(0, 0) = +4. The learning rate, rj 9 is 0.5. 

4. Experimental Results 

We have tested our system on two sets of fingerprint im- 
ages. Set 1 contains 10 images (380 x 380) per finger from 
18 individuals for a total of 180 images, which were cap- 
tured with a scanner manufactured by Identix. Set 2 con- 
tains 10 images (640 x 480) per finger from 30 individu- 
als for a total of 300 images, which were captured with a 
scanner manufactured by Digital Biometrics. The captured 
fingerprint images vary in quality. Approximately 90% are 
of satisfactory quality, while about 10% are of poor quality. 





(a) Identix ' 



(b) Digital Biometrics 



Figure 6. Distributions of correct and incor- 
rect matching scores: vertical axis repre- 
sents distribution of matching scores in per- 
centage. 



4.1. Matching 

Each fingerprint in the test set was matched with the 
other fingerprints in the set. A matching was labeled cor- 
rect if the matched fingerprint was among the 9 other fin- 
gerprints of the same individual, and incorrect otherwise. 
The distributions of matching scores are shown in Figure 6. 
This result indicates that our algorithm is capable of effec- 
tively differentiating fingerprints by setting an appropriate 
value of the matching threshold. Table 1 shows the recog- 
nition rates and reject rates with different threshold values 



on the matching score. We have observed that the incorrect 
matchings occur mainly due to fingerprint images with poor 
quality. 



Threshold 
Value 


Recogni- 
tion Rate 
(Set 1) 


Reject 
Rate 
(Set 1) 


Recogni- 
tion Rate 
(Set 2) 


Reject 
Rate 
(Set 2) 


20 


99.84% 


11.23% 


99.49% 


8.67% 


24 


99.98% 


16.50% 


99.90% 


14.06% 


28 


99.99% 


25.22% 


99.99% 


18.85% 


32 


100% 


27.72% 


99.99% 


22.08% 



Table 1. Recognition and reject rates on test 
sets with different threshold values 



4.2. Verification 

In on-line verification, a user indicates his/her identity. 
Therefore, the system matches the input fingerprint image 
only to his/her stored template. To determine the verifi- 
cation accuracy of our system, we used each one of our 
database images as template and all the others as input fin- 
gerprints which need to be verified. An input fingerprint is 
matched against the template. If the matching score is be- 
low the threshold value of 25, then the input fingerprint is 
rejected. If the matching score exceeds 25, then a valid ver- 
ification is established. With this scheme, based on 89,700 
(300 x 299) verifications, a 99.99% verification rate can be 
achieved with a tolerable reject rate (15%). 

In order for an on-line fingerprint verification system to 
be acceptable in practice, the response time of the system 
needs to be within a few seconds. Table 2 shows that our on- 
line fingerprint verification system does meet the response 
time requirement of on-line verification. 



Minutia Extraction 
(second) 


Minutia Matching 
(second) 


Total 
(second) 


5.35 


2.55 


7.90 



Table 2. Average CPU time for minutia extrac- 
tion and matching on a SPARC 20 worksta- 
tion. 



5. Conclusions 

We have introduced an on-line fingerprint verification 
system. The implemented minutia extraction algorithm is 
accurate and fast in minutia extraction. The alignment- 
based elastic matching algorithm is capable of finding the 



correspondences between minutiae without resorting to ex- 
haustive search. It provides a good performance, because 
it has the ability to adaptively compensate for the nonlin- 
ear deformations and inexact pose transformations between 
different fingerprints. Experimental results show that our 
system achieves excellent performance in a real on-line ver- 
ification environment. It meets the response time require- 
ments of on-line verification with high accuracy. 

Based on the experimental results, we observe that the 
matching errors of our system mainly result from (i) incor- 
rect minutiae and (ii) inaccurate alignment. A number of 
factors are detrimental to the correct location of minutia. 
Among them, poor image quality is the most serious one. 
We are currently investigating a number of image enhance- 
ment schemes. 
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Abstract 

Fingerprint verification is one of the most reliable personal identification methods. 
However, manual fingerprint verification is so tedious, time-consuming, and expen- 
sive that it is incapable of meeting today's increasing performance requirements. An 
automatic fingerprint identification system (AFIS) is widely needed. It plays a very im- 
portant role in forensic and civilian applications such as criminal identification, access 
control, and ATM card verification. This paper describes the design and implementation 
of an on-line fingerprint verification system which operates in two stages: (i) minutia 
extraction and (ii) minutia matching. An improved version of the minutia extraction 
algorithm proposed by Ratha et al. ; which is much faster and more reliable, is imple- 
mented for extracting features from an input fingerprint image captured with an on-line 



inkless scanner. For minutia matching, an alignment-based elastic matching algorithm 
has been developed. This algorithm is capable of finding the correspondences between 
minutiae in the input image and the stored template without resorting to exhaustive 
search and has the ability of adaptively compensating for the nonlinear deformations 
and inexact pose transformations between fingerprints. The system has been tested on 
two sets of fingerprint images captured with inkless scanners. The verification accuracy 
is found to be acceptable. Typically, a complete fingerprint verification procedure takes, 
on an average, about 8 seconds on a SPARC 20 workstation. These experimental re- 
sults show that our system meets the response time requirements of on-line verification 
with high accuracy. 

Keywords: biometrics, fingerprints, matching, verification, minutia, orientation field, 
ridge extraction. 

1 Introduction 

Fingerprints are graphical flow-like ridges present on human fingers. They have been widely 
used in personal identification for several centuries [11]. The validity of their use has been 
well established. Inherently, using current technology fingerprint identification is much more 
reliable than other kinds of popular personal identification methods based on signature, face, 
and speech [11, 3, 15]. Although fingerprint verification is usually associated with criminal 
identification and police work, it has now become more popular in civilian applications such 
as access control, financial security and verification of firearm purchasers and driver license 
applicants [11, 3]. Usually, fingerprint verification is performed manually by professional 
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fingerprint experts. However, manual fingerprint verification is so tedious, time-consuming, 
and expensive that it does not meet the performance requirements of the new applications. 
As a result, automatic fingerprint identification systems (AFIS) are in great demand [11]. 
Although significant progress has been made in designing automatic fingerprint identification 
systems over the past thirty years, a number of design factors (lack of reliable minutia ex- 
traction algorithms, difficulty in quantitatively defining a reliable match between fingerprint 
images, fingerprint classification, etc.) create bottlenecks in achieving the desired perfor- 
mance [11]. 




(a) (b) 

Figure 1: Inkless fingerprint scanners: (a) manufactured by Identix; (b) manufactured by 
Digital Biometrics. 

An automatic fingerprint identification system is concerned with some or all of the fol- 
lowing issues: 

• Fingerprint acquisition. How to acquire fingerprint images and how to represent them 
in a proper format? 

• Fingerprint verification. To determine whether two fingerprints are from the same 
finger. 

• Fingerprint identification. To search for a query fingerprint in a database. 
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• Fingerprint classification. To assign a given fingerprint to one of the pre-specified 
categories according to its geometric appearance. 

A number of methods are used to acquire fingerprints. Among them, the inked impression 
method remains the most popular. It has been essentially a standard technique for fingerprint 
acquisition for more than a hundred years [3]. The first step in capturing an inked impression 
of a fingerprint is to place a few dabs of ink on a slab and rolling it out smoothly with a roller 
until the slab is covered with a thin, even layer of ink. Then the finger is rolled from one side of 
the nail to the other side over the inked slab which inks the ridge patterns on top of the finger 
completely. After that, the finger is rolled on a piece of paper so that the inked impression 
of the ridge pattern of the finger appears on the paper. Obviously, this method is time- 
consuming and unsuitable for an on-line fingerprint verification system. Inkless fingerprint 
scanners are now available which are capable of directly acquiring fingerprints in digital 
form. This method eliminates the intermediate digitization process of inked fingerprint 
impressions and makes it possible to build an on-line system. Figure 1 shows the two inkless 
fingerprint scanners used in our verification system. Fingerprint images captured with the 
inked impression method and the inkless impression method are shown in Figure 2. 

The goal of fingerprint classification is to assign a given fingerprint to a specific category 
according to its geometric properties (Figure 3 shows a coarse-level fingerprint classifica- 
tion). The main purpose of fingerprint classification is to facilitate the management of large 
fingerprint databases and to speedup the process of fingerprint matching. Generally, manual 
fingerprint classification is performed within a specific framework such as the well-known 
Henry system [3]. Different frameworks use different sets of properties. However, no matter 
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Figure 2: Comparison of fingerprint images captured with (a) inked impression method (from 
NIST database) and (b) inkless impression method (with a scanner manufactured by Digital 
Biometrics). 



what type of framework is used, the classification is based on ridge patterns, local ridge ori- 
entations and minutiae. Therefore, if these properties can be described quantitatively and 
extracted automatically from a fingerprint image then fingerprint classification will become 
an easier task. During the past several years, a number of researchers have attempted to 
solve the fingerprint classification problem [11, 3, 9, 10, 26]. Unfortunately, their efforts have 
not resulted in the desired accuracy. Algorithms reported in the literature classify finger- 
prints into 5 or 6 categories with about 90% classification accuracy on a medium size test 
set (several thousand images) [9, 10, 26]. However, to achieve a higher recognition accuracy 
with a large number of categories still remains a difficult problem. 

Fingerprint verification determines whether two fingerprints are from the same finger 
or not. It is widely believed that if two fingerprints are from the same source, then their 
local ridge structures (minutia details) match each other topologically [11, 3]. Eighteen 
different types of local ridge descriptions have been identified [11]. The two most prominent 
structures are ridge endings and ridge bifurcations which are usually called minutiae. Figure 4 
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(a) (b) (c) 




(d) (e) (f) 

Figure 3: A coarse-level fingerprint classification of six categories: (a) arch, (b) tented arch, 
(c) right loop, (d) left loop, (e) whorl, and (f) twin loop. 




Ridge Ending Ridge Bifurcation 

Figure 4: Ridge ending and ridge bifurcation. 

shows examples of ridge endings and ridge bifurcations. Based on this observation and by 
representing the minutiae as a point pattern, an automatic fingerprint verification problem 
may be reduced to a point pattern matching (minutia matching) problem. In the ideal 
case, if (i) the correspondences between the template and input fingerprint are known, (ii) 
there are no deformations such as translation, rotation and nonlinear deformations, etc. 
between them, and (iii) each minutia present in a fingerprint image is exactly localized, 
then fingerprint verification consists of the trivial task of counting the number of spatially 
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Figure 5: Two different fingerprint images from the same finger. In order to know the 
correspondence between the minutiae of these two fingerprint images, all the minutiae must 
be precisely localized and the deformations must be recovered. 

matching pairs between the two images. However, in practice (i) no correspondence is 
known beforehand, (ii) there are relative translation, rotation and nonlinear deformations 
between template minutiae and input minutiae, (iii) spurious minutiae are present in both 
templates and inputs, and (iv) some minutiae are missed. Therefore, in order for a fingerprint 
verification algorithm to operate under such circumstances, it is necessary to automatically 
obtain minutia correspondences, to recover deformations, and to detect spurious minutiae 
from fingerprint images. Unfortunately, this goal is quite difficult to achieve. Figure 5 
illustrates the difficulty with an example of two fingerprint images of the same finger. 

Fingerprint identification refers to the process of matching a query fingerprint against a 
given fingerprint database to establish the identity of an individual. Its goal is to quickly 
determine whether a query fingerprint is present in the database and to retrieve those which 
are most similar to the query from the database. The critical issues here are both retrieval 
speed and accuracy. In fact, this problem relates to a number of techniques studied under 
the auspices of computer vision, pattern recognition, database, and parallel processing. Op- 
erational fingerprint retrieval systems are being used by various law enforcement agencies 

[ii]. 
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In this paper, we will introduce an on-line fingerprint verification system whose purpose 
is to capture fingerprint images using an inkless scanner and to compare them with those 
stored in the database in "real time" . Such a system has great utility in a variety of personal 
identification and access control applications. The overall block diagram of our system is 
shown in Figure 6. It operates as follows: (i) off-line phase: several impressions (depending 
on the specification of the system) of the fingerprint of a person to be verified are first 
captured and processed by a feature extraction module; the extracted features are stored as 
templates in a database for later use; (ii) on-line phase: the individual to be verified gives 
his/her identity and places his/her finger on the inkless fingerprint scanner, minutia points 
are extracted from the captured fingerprint image; these minutiae are then fed to a matching 
module, which matches them against his/her own templates in the database. 



Off-line processing 
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Figure 6: Overview of our on-line fingerprint verification system. 

The following two modules are the main components of our on-line fingerprint verification 
system: 



• Minutiae extraction. Minutiae are ridge endings or ridge bifurcations. Generally, if 
a perfect segmentation can be obtained, then minutia extraction is just a trivial task 
of extracting singular points in a thinned ridge map. However, in practice, it is not 
always possible to obtain a perfect ridge map. Some global heuristics need to be used 
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to overcome this limitation. 

• Minutia matching. Minutia matching, because of deformations in sensed fingerprints, 
is an elastic matching of point patterns without knowing their correspondences before- 
hand. Generally, finding the best match between two point patterns is intractable even 
if minutiae are exactly located and no deformations exist between these two point pat- 
terns. The existence of deformations makes the minutia matching much more difficult. 

For segmentation and minutia extraction, a modified version of the minutia extraction 
algorithm proposed in [18] is implemented which is much faster and more reliable for minutia 
extraction. We propose a hierarchical approach to obtain a smooth orientation field estimate 
of the input fingerprint image, which greatly improves the performance of minutia extraction. 
For minutia matching, we propose an alignment-based elastic matching algorithm. This 
algorithm is capable of finding the correspondences between minutiae without resorting to an 
exhaustive search and has the ability to adaptively compensate for the nonlinear deformations 
and inexact pose transformations between different fingerprints. Experimental results show 
that our system achieves excellent performance in a real environment. 

In the following sections we will describe in detail our on-line fingerprint verification 
system. Section 2 mainly discusses the fingerprint feature extraction module. Section 3 
presents our minutia matching algorithm. Experimental results on two fingerprint databases 
captured with two different inkless scanners are described in section 4. Section 5 contains 
the summary and discussion. 
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2 Minutia Extraction 



It is widely known that a professional fingerprint examiner relies on minute details of ridge 
structures to make fingerprint identifications [11, 3]. The topological structure of the minu- 
tiae of a fingerprint is unique and invariant with aging and impression deformations [11, 3]. 
This implies that fingerprint identification can be based on the topological structural match- 
ing of these minutiae. This reduces the complex fingerprint verification to minutia matching 
process which, in fact, is a sort of point pattern matching with the capability of tolerating, 
to some restricted extent, deformations of the input point patterns. Therefore, the first stage 
in an automatic fingerprint verification procedure is to extract minutiae from fingerprints. 
In our on-line fingerprint verification system, we have implemented a minutia extraction 
algorithm which is an improved version of the method proposed by Ratha et al. [18]. Its 
overall flowchart is depicted in Figure 7. We assume that the resolution of input fingerprint 
images is 500 dpi. 
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Figure 7: Flowchart of the minutia extraction algorithm. 



2.1 Estimation of Orientation Field 

A number of methods have been proposed to estimate the orientation field of flow-like pat- 
terns [17]. In our system, a new hierarchical implementation of Rao's algorithm [17] is used. 
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Rao's algorithm consists of the following main steps: 

1. Divide the input fingerprint image into blocks of size W x W. 

2. Compute the gradients G x and G y at each pixel in each block. 

3. Estimate the local orientation of each block using the following formula: 

a -K^-U S%iSEi2C,(i,j)C,(»,j) x 

where W is the size of the block, and G x and G y are the gradient magnitudes in x and 
y directions, respectively. 

The orientation field of a good quality fingerprint image can be reasonably estimated with 
this algorithm. However, the presence of high-curvature ridges, noise, smudges, and breaks 
in ridges leads to a poor estimate of the local orientation field. A post- processing procedure 
needs to be applied to overcome this limitation. In our system, the following iterative steps 
are added to improve an inconsistent orientation field: 

• Compute the consistency level of the orientation field in the local neighborhood of a 
block (ijj) with the following formula: 

Co = -J=/ £ \e(i',r)-0(i,j)f, (2) 



(»'J')€D 



\0'-9\ = < 



d tf(d={9'-9 + 360) mod 360) < 180, 

(3) 

d - 180 otherwise, 



where D represents the local neighborhood around the block (i, j) (in our system, the 
size of D is 5 x 5); N is the number of blocks within D\ O^'.j 1 ) and 0(ij) are local 
ridge orientations at blocks (i', f) and (i, j), respectively. 
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• If the consistency level (Eq.(2)) is above a certain threshold T c , then the local orien- 
tations around this region are re-estimated at a lower resolution level until it is below 
a certain level. With this post-smoothing scheme, a fairly smooth orientation field 
estimate can be obtained. Figure 8 shows the orientation field of a fingerprint image 
estimated with our new algorithm. 




(a) Rao's method (b) Hierarchical method 



Figure 8: Comparison of orientation fields by Rao's method and the proposed hierarchical 
method; the block size (W x W) is 16 x 16 and the size of D is 5 x 5. 

After the orientation field of an input fingerprint image is estimated, a segmentation 
algorithm which is based on the local variance of grey level is used to locate the region of 
interest from the fingerprint image. In our segmentation algorithm, we assume that there is 
only one fingerprint present in the image. 

2.2 Ridge Detection 

After the orientation field of the input image is estimated and the fingerprint region is 
located, the next step of our minutia exaction algorithm is ridge detection. The most salient 
property corresponding to ridges in a fingerprint image is the fact that grey level values 
on ridges attain their local maxima along the normal directions of local ridges. Therefore, 
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pixels can be identified to be ridge pixels based on this property. In our minutia detection 
algorithm, a fingerprint image is first convolved with the following two masks, h t (x,y\u,v) 
and hb(x,y]u,v), of size L x H (11 x 7 in our system), respectively. These two masks are 
capable of adaptively accentuating the local maximum grey level values along the normal 
direction of the local ridge direction: 



h t {x,y\u,v) = i 



-vks e ^> [{u = fotan(g(s >y )) - 2cM g (xM ),v € Q 
^e-£, if u= {vtzn(9(x,y))),ven 
0, otherwise, 

-^ e "^> if u = ("tan(g(a,y)) + 2cot $ (sjl)) ),v £ fl 
^e"^, iiu=(vtan(6(x,y))),veSl 
otherwise, 



h b (x,y;u,v) = < 



CI = 



(4) 



0, 



Lan(6(x,y)) 



Lsin(6(x, y)) 



(5) 



(6) 



where 9{x,y) represents the local ridge direction at pixel (x 9 y). If both the grey level values 
at pixel (x,y) of the convolved images are larger than a certain threshold T ridge , then pixel 
(x, y) is labeled as a ridge. By adapting the mask width to the width of the local ridge, this 
algorithm can efficiently locate the ridges in a fingerprint image. 

However, due to the presence of noise, breaks, and smudges, etc. in the input image, the 
resulting binary ridge map often contains holes and speckles. When ridge skeletons are used 
for the detection of minutiae, the presence of such holes and speckles will severely handicap 
the performance of our minutia extraction algorithm because these holes and speckles may 
drastically change the skeleton of the ridges. Therefore, a hole and speckle removal procedure 
needs to be applied before ridge thinning. 
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After the above steps are performed on an input fingerprint image, a relatively smooth 
ridge map of the fingerprint is obtained. The next step of our minutia detection algorithm 
is to thin the ridge map and locate the minutiae. 

2,3 Minutia Detection 

Minutia detection is a trivial task when an ideal thinned ridge map is obtained. Without a 
loss of generality, we assume that if a pixel is on a thinned ridge (8-connected), then it has 
a value 1, and 0 otherwise. Let (x, y) denote a pixel on a thinned ridge, and N 0: N U ...,N 7 
denote its 8 neighbors. A pixel (x, y) is a ridge ending if (£? =0 N { ) = 1 and a ridge bifurcation 
if (ELo Ni) > 2. However, the presence of undesired spikes and breaks present in a thinned 
ridge map may lead to many spurious minutiae being detected. Therefore, before the minutia 
detection, a smoothing procedure is applied to remove spikes and to join broken ridges. Our 
ridge smoothing algorithm uses the following heuristics: 

• If a branch in a ridge map is roughly orthogonal to the local ridge directions and its 
length is less than a specified threshold T 6 , then it will be removed. 

• If a break in a ridge is short enough and no other ridges pass through it, then it will 
be connected. 

Although the above heuristics do delete a large percentage of spurious minutiae, many spu- 
rious minutiae still survive. The reason is that the above processing relies on local ridge 
information. If this information itself is unreliable, then the above heuristics have no way 
of differentiating false minutiae from true minutiae. Therefore, a refinement which is based 
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on structural information is necessary. Our refinement algorithm eliminates the spurious 
minutiae based on the following rules: 

• If several minutiae form a cluster in a small region, then remove all of them except for 
the one nearest to the cluster center. 

• If two minutiae are located close enough, facing each other, but no ridges lie between 
them, then remove both of them. 

After the above refinement procedure is performed, the surviving minutiae are treated as true 
minutiae. Although the above heuristics can not ensure a perfect location of each minutia, 
they are able to delete several spurious minutiae. For each surviving minutia, the following 
parameters are recorded: (i) x-coordinate, (ii) y-coordinate, (iii) orientation which is defined 
as the local ridge orientation of the associated ridge, and (iv) the associated ridge. The 
recorded ridges are represented as one-dimensional discrete signals which are normalized 
by the average inter-ridge distance. These recorded ridges are used for alignment in the 
minutia matching phase. Figure 9 shows the results of our minutia extraction algorithm on 
a fingerprint image captured with an inkless scanner. 

3 Minutia Matching 

Generally, an automatic fingerprint verification/identification is achieved with point pattern 
matching (minutiae matching) instead of a pixel-wise matching or a ridge pattern matching 
of fingerprint images. A number of point pattern matching algorithms have been proposed 
in the literature [23, 1, 21, 16]. Because a general point matching problem is essentially 
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(a) input image (b) orientation field 




(e) thinned ridge map (f ) extracted minutiae 



Figure 9: Results of our minutia extraction algorithm on a fingerprint image (512 x 512) 
captured with an inkless scanner; (a) input image; (b) orientation field superimposed on the 
input image; (c) fingerprint region; (d) extracted ridges; (e) thinned ridge map; (f) extracted 
minutiae and their orientations superimposed on the input image. 
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intractable, features associated with each point and their spatial properties such as the 
relative distances between points are often used in these algorithms to reduce the exponential 
number of search paths. 

The relaxation approach [16] iteratively adjusts the confidence level of each corresponding 
pair based on its consistency with other pairs until a certain criterion is satisfied. Although 
a number of modified versions of this algorithm have been proposed to reduce the matching 
complexity [23], these algorithms are inherently slow because of their iterative nature. 

The Hough transform-based approach proposed by Stockman et al. [22] converts point 
pattern matching to a problem of detecting the highest peak in the Hough space of trans- 
formation parameters. It discretizes the transformation parameter space and accumulates 
evidence in the discretized space by deriving transformation parameters that relate two point 
patterns using a substructure or feature matching technique. Karu and Jain [8] proposed 
a hierarchical Hough transform-based registration algorithm which greatly reduced the size 
of accumulator array by a multi-resolution approach. However, if the number of minutia 
point is less than 30, then it is very difficult to accumulate enough evidence in the Hough 
transform space for a reliable match. 

Another approach to point matching is based on energy minimization. This approach 
defines a cost function based on an initial set of possible correspondences and uses an appro- 
priate optimization algorithm such as genetic algorithm [1] and simulated annealing [21] to 
find a possible suboptimal match. These methods tend to be very slow and are unsuitable 
for an on-line fingerprint verification system. 

In our system, an alignment-based matching algorithm is implemented. Recognition by . 
alignment has received a great deal of attention during the past few years [12], because 
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it is simple in theory, efficient in discrimination, and fast in speed. Our alignment-based 
matching algorithm decomposes the minutia matching into two stages: (i) Alignment stage, 
where transformations such as translation, rotation and scaling between an input and a 
template in the database are estimated and the input minutiae are aligned with the template 
minutiae according to the estimated parameters; and (ii) Matching stage, where both the 
input minutiae and the template minutiae are converted to polygons in the polar coordinate 
system and an elastic string matching algorithm is used to match the resulting polygons. 

3.1 Alignment of Point Patterns 

Ideally, two sets of planar point patterns can be aligned completely by two corresponding 
point pairs. A true alignment between two point patterns can be obtained by testing all 
possible corresponding point pairs and selecting the optimal one. However, due to the 
presence of noise and deformations, the input minutiae cannot always be aligned exactly 
with respect to those of the templates. In order to accurately recover pose transformations 
between two point patterns, a relatively large number of corresponding point pairs need 
to be used. This leads to a prohibitively large number of possible correspondences to be 
tested. Therefore, an alignment by corresponding point pairs is not practical even though it 
is feasible. 

It is well known that corresponding curve segments are capable of aligning two point 
patterns with a high accuracy in the presence of noise and deformations. Each minutia in 
a fingerprint is associated with a ridge. It is clear that a true alignment can be achieved 
by aligning corresponding ridges (see Figure 10). During the minutiae detection stage, 
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Figure 10: Alignment of the input ridge and the template ridge. 

when a minutia is extracted and recorded, the ridge on which it resides is also recorded. 
This ridge is represented as a planar curve with its origin coincident with the minutia and 
its x-coordinate being in the same direction as the direction of the minutia. Also, this 
planar curve is normalized with the average inter-ridge distance. By matching these ridges, 
the relative pose transformation between the input fingerprint and the template can be 
accurately estimated. To be specific, let R d and R D denote the sets of ridges associated with 
the minutiae in input image and template, respectively. Our alignment algorithm can be 
described in terms of the following steps: 

1. For each ridge d G represent it as an one-dimensional discrete signal and match it 
against each ridge, D G R D according to the following formula: 

where L is the minimal length of the two ridges and d{ and D { represent the distances 
from point i on the ridges d and D to the x-axis, respectively. The sampling interval on 
a ridge is set to the average inter-ridge distance. If the matching score 5 (0 < S < 1) 
is larger than a certain threshold T r , then go to step 2, otherwise continue to match 
the next pair of ridges. 
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2. Estimate the pose transformation between the two ridges (Figure 10). Generally, a 
least-square method can be used to estimate the pose transformation. However, in 
our system, we observe that the following method is capable of achieving the same 
accuracy with less computation. The translation vector (Ax, At/) T between the two 
corresponding ridges is computed by 
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where {x d ,y d ) T and (x D , y D ) T are the x and y coordinates of the two minutiae, which 
are called reference minutiae, associated with the ridges d and D, respectively. The 
rotation angle A# between the two ridges is computed by 

A0 = y£(7i-lU (9) 

where L is the minimal length of the two ridges d and D\ 7* and T { are radial angles 
of the ith point on the ridge with respect to the reference minutia associated with the 
two ridges d and D, respectively. The scaling factor between the input and template 
images is assumed to be 1. This is reasonable, because fingerprint images are captured 
with the same device in both the off-line processing phase and the on-line verification 
phase. 

3. Denote the minutia (x d , y d , Q d ) T , based on which the pose transformation parameters 
are estimated, as the reference minutia. Translate and rotate all the N input minutiae 
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with respect to this reference minutia, according to the following formula: 
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where (a;*, y*, 0j) T , (i = 1,2, ...,JV), represents an input minutia and {xf.y^O^) 7 rep- 
resents the corresponding aligned minutia. 

3,2 Aligned Point Pattern Matching 

If two identical point patterns are exactly aligned with each other, each pair of corresponding 
points are completely coincident. In such a case, a point pattern matching can be simply 
achieved by counting the number of overlapping pairs. However, in practice, such a situation 
is not encountered. On the one hand, the error in determining and localizing minutia hinders 
the alignment algorithm to recover the relative pose transformation exactly, while on the 
other hand, our alignment scheme described above does not model the nonlinear deformation 
of fingerprints which is an inherent property of fingerprint impressions. With the existence 
of such a nonlinear deformation, it is impossible to exactly recover the position of each input 
minutia with respect to its corresponding minutia in the template. Therefore, the aligned 
point pattern matching algorithm needs to be elastic which means that it should be capable 
of tolerating, to some extent, the deformations due to inexact extraction of minutia positions 
and nonlinear deformations. Usually, such an elastic matching can be achieved by placing 
a bounding box around each template minutia, which specifies all the possible positions of 
the corresponding input minutia with respect to the template minutia, and restricting the 
corresponding minutia in the input image to be within this box [18]. This method does 
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not provide a satisfactory performance in practice, because local deformations may be small 
while the accumulated global deformations can be quite large. We have implemented an 
adaptive elastic matching algorithm with the ability to compensate the minutia localization 
errors and nonlinear deformations. 

Let P - {{x{ ,yf,0f) T , {xm>Vmi8m) T ) denote the set of M minutiae in the template 
and Q = ({xf ,y? ,9?) T , {x%,y%,9%) T ) denote the set of N minutiae in the input image 
which is aligned with the above template with respect to a given reference minutia point. 
The steps in our elastic point pattern matching algorithm are given below: 

1. Convert each minutia point to the polar coordinate system with respect to the corre- 
sponding reference minutia on which the alignment is performed: 
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where (x*,y*,6?) T are the coordinates of a minutia, {x T ,y T ,6 T ) T are the coordinates 
of the reference minutia, and {r u e u 9i) T is the representation of the minutia in polar 
coordinate system (r, represents the radial distance, e { represents the radial angle and 
6i represents the orientation of the minutia with respect to the reference minutia). 

2. Represent the template and the input minutiae in the polar coordinate system as 
symbolic strings by concatenating each minutia in the increasing order of radial angles: 



Q P = ((r?,ef,ef) T ,..: ) (r Q N ,el0%n 



(12) 
(13) 
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where (rf , ef , 0f ) and (r?, e?, 5?) represent the corresponding radius, radial angle, 
and normalized minutia orientation with respect to the reference minutia, respectively. 

3. Match the resulting strings P p and Q p with a dynamic-programming algorithm [4] to 
find the edit distance between P p and Q p which is described below. 

4. Use the edit distance between P p and Q p to establish the correspondence of the minutiae 
between P p and Q p . The matching score, M pqy is then computed according to the 
following formula: 

= max{M, N} ' (14) 

where N pa i r is the number of the minutiae which fall in the bounding boxes of template 
minutiae. The maximum and minimum values of the matching score are 100 and 1, 
respectively. The former value indicates a perfect match, while the later value indicates 
no match at all. 

Minutia matching in the polar coordinate has several advantages. We have observed that the 
nonlinear deformation of fingerprints has a radial property. In other words, the nonlinear 
deformation in a fingerprint impression usually starts from a certain point (region) and 
nonlinearly radiates outward. Therefore, it is beneficial to model it in the polar space. At 
the same time, it is much easier to formulate rotation, which constitutes the main part of 
the alignment error between an input image and a template, in the polar space than in the 
Cartesian space. The symbolic string generated by concatenating points in an increasing 
order of radial angle in polar coordinate uniquely represents a point pattern. This reveals 
that the point pattern matching can be achieved with a string matching algorithm. 
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A number of string matching algorithms have been reported in the literature [4]. Here, 
we are interested in incorporating an elastic criteria into a string matching algorithm. Gen- 
erally, string matching can be thought of as the maximization/minimization of a certain cost 
function such as the edit distance. Intuitively, including an elastic term in the cost function 
of a string matching algorithm can achieve a certain amount of error tolerance. Given two 
strings P p and Q p of lengths M and N t respectively, the edit distance, C(M,7V), in our 
algorithm is recursively defined with the following equations: 



C(m, n) = < 



mm < 



► 0 < m < M and 0 < n < JV, 
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0 if ra = 0 or n = 0 

C(m- l,n) + fi 
C(m,n - 1) + 
C(m - 1, n - 1) + w(m, n) 

<* \ r m - r%\ + (3Ae + jA6 if |r£ - r«| < 6, Ae < e and A0 < e 

otherwise, 

a if (a = (e£ - e% + 360) mod 360) < 180 

a — 180 otherwise, 

a if (a = (0£ - ^ + 360) mod 360) < 180 

a - 180 otherwise, 

where a, /?, and 7 are the weights associated with each component, respectively; 5, e and 
€ specify the bounding box; and Q is a pre-specified penalty for a mismatch. Such an edit 
distance, to some extent, captures the elastic property of string matching. It represents a 
cost of changing one polygon to the other. However, this scheme can only tolerate, but 
not compensate for, the adverse effect on matching produced by the inexact localization of 
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minutia and nonlinear deformations. Therefore, an adaptive mechanism is needed. This 
adaptive mechanism should be able to track the local nonlinear deformation and inexact 
alignment and try to alleviate them during the minimization process. However, we do not 
expect that this adaptive mechanism can handle the "order flip" of minutiae, which, to some 
extent, can be solved by an exhaustive re-ordering and matching within a local angular 
window. 

In our matching algorithm, the adaptation is achieved by adjusting the bounding box 
(Figure 11) when an inexact match is found during the matching process. It can be repre- 
sented as follows: 

6i(m,n) < (r£ - r£) < 6 h (m,n) 
€/(m, n)) < Ae < e/,(m,n) 
A6<e 

otherwise, 

<5,(m,n) < (r£ - rj?) < S h (m,n) 
ei(m,n)) < Ae < e h {m,n) 
A9<e 
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where K/(m,ra) represents the penalty for matching a pair of minutiae (r£,e£,0£) r and 
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Figure 11: Bounding box and its adjustment. 



( r n > e ny6n) T i M m > n )> Sh(m 9 n), (ra, n), and e/^m, n) specify the adaptive bounding box in 
the polar coordinate system (radius and radial angle); and 77 is the learning rate. This elastic 
string matching algorithm has a number of parameters which are critical to its performance. 
We have empirically determined the values of these parameters as follows: 5/(0,0) = -8; 
^(0,0) = +8; e,(0,0) = -7.5; c A (0,0) = +7.5; e = 30; a = 1.0; (3 = 2.0; 7 = 0.1; 
Q = 200(a + (3 + 7); 7] = 0.5. The values of 5/(0, 0), 6 h (0, 0), e,(0, 0), and c A (0, 0) depend on 
the resolution of fingerprint images. Figure 12 shows the results of applying the matching 
algorithm to an input minutia set and a template. 

4 Experimental Results 

We have tested our on-line fingerprint verification system on two sets of fingerprint images 
captured with two different inkless fingerprint scanners. Set 1 contains 10 images per finger 
from 18 individuals for a total of 180 fingerprint images, which were captured with a scanner 
manufactured by Identix. The size of these images is 380 x 380. Set 2 contains 10 images per 
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(a) 



(b) 




(c) 



(d) 



Figure 12: Results of applying the matching algorithm to an input minutia set and a tem- 
plate; (a) input minutia set; (b) template minutia set; (c) alignment result based on the 
minutiae marked with green circles; (d) matching result where template minutiae and their 
correspondences are connected by green lines. 

finger from 61 individuals for a total of 610 fingerprint images, which were captured with a 
scanner manufactured by Digital Biometrics. The size of these images is 640 x 480. When 
these fingerprint images were captured, no restrictions on the position and orientation of 
fingers were imposed. The captured fingerprint images vary in quality. Figures 13 and 14 
show some of the fingerprint images in our database. Approximately 90% of the fingerprint 
images in our database are of reasonable quality similar to those shown in Figures 13 and 14, 
while about 10% of the fingerprint images in our database are not of good quality (Figure 15), 
which are mainly due to large creases and smudges in ridges and dryness of the impressed 
finger. First, we report some initial results on fingerprint matching, followed by fingerprint 
verification. The reasons why we did not use NIST-4 fingerprint database [25] to test the 
performance of our system are as follows: (i) we concentrate on live-scan verification, and 
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(ii) NIST-4 fingerprint database is a very difficult fingerprint database which contains a 
large number of fingerprint images of poor quality and no result has been reported from 



other on-line verification systems for comparison. 




Figure 14: Fingerprint images captured with a scanner manufactured by Digital Biometrics; 
the size of these images is 640 x 480; all the three images are from the same individual's 
finger. 




Figure 15: Fingerprint images of poor quality. 
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Normalized Matching Score Normatized Matching Score 



(a) Identix (b) Digital Biometrics 

Figure 16: Distributions of correct and incorrect matching scores; vertical axis represents 
distribution of matching scores in percentage; (a) distribution of matching scores on test set 
1 (180 images); (b) distribution of matching scores on test set 2 (610 images). 

4.1 Matching 

Each fingerprint in the test set was matched with the other fingerprints in the set. A 
matching was labeled correct if the matched fingerprint was among the 9 other fingerprints 
of the same individual, and incorrect otherwise. A total of 32,220 (180 x 179) matchings 
have been performed on test set 1 and 371,490 (610 x 609) matchings on test set 2. The 
distributions of correct and incorrect matching scores are shown in Figure 16. It can be 
seen from this figure that there exist two peaks in the distribution of matching scores. One 
pronounced peak corresponds to the incorrect matching scores which is located at a value 
around 10, and the other peak which resides at a value of 40 is associated with the correct 
matching scores. This indicates that our algorithm is capable of differentiating fingerprints 
at a high correct rate by setting an appropriate value of the threshold. Table 1 shows the 
verification rates and reject rates with different threshold values. The reject rate is defined 
as the percentage of correct fingerprints with their matching scores below the threshold 
value. As we have observed, both the incorrect matches and the high reject rates are due 
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to fingerprint images with poor quality such as those shown in Figure 15. We can improve 
these matching results by ensuring that the database does not contain such poor quality 
fingerprint images. 



Threshold 
Value 


Verification 
Rate 


Reject 
Rate 


20 


99.839% 


11.23% 


22 


99.947% 


13.33% 


24 


99.984% 


16.48% 


26 


99.994% 


20.49% 


28 


99.996% 


25.19% 


30 


100% 


27.72% 



(a) 



Threshold 
Value 


Verification 
Rate 


Reject 
Rate 


20 


99.426% 


11.23% 


22 


99.863% 


14.55% 


24 


99.899% 


16.78% 


26 


99.969% 


20.20% 


28 


99.989% 


23.15% 


30 


99.999% 


27.45% 



(b) 



Table 1: The verification rates and reject rates on test sets with different threshold values; 
(a) using Identix system (180 images); (b) using Digital Biometrics system (610 images). 



4.2 Verification 

In on-line verification, a user indicates his/her identity. Therefore, the system matches 
the input fingerprint image only to his/her stored templates. To determine the verification 
accuracy of our system, we used each one of our database images as an input fingerprint 
which needs to be verified. An input fingerprint image was matched against all the 9 other 
images of the same finger. If more than one half of the 9 matching scores exceeded the 
threshold value of 25, then the input fingerprint image is said to be from the same finger as 
the templates and a valid verification is established. With this scheme, a 100% verification 
rate can be achieved with a reject rate around 16% on both test sets. Again, this reject 
rate can be reduced by preprocessing the database to remove the stored templates of poor 
quality. This demonstrates that, in practice, using a k-nearest neighbor type of matching 
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is adequate for a successful verification. Table 2 shows the matching rate which is defined 
as the percentage of the correct fingerprints (of the same finger) present among the best n 
(n = 1, ...9) matches. 

For an on-line fingerprint verification system to be acceptable in practice, its response 
time needs to be within a few seconds. Table 3 shows the CPU requirements of our sys- 
tem. The CPU time for one verification, including fingerprint image acquisition, minutia 
extraction and minutia matching, is, on an average, approximately 8 seconds on a SPARC 
20 workstation. It indicates that our on-line fingerprint verification system does meet the 
response time requirement of on-line verification. 

The number of tests done on an automatic fingerprint identification system is never 
enough. Performance measures are as much a function of the algorithm as they are a func- 
tion of the database used for testing. The biometrics community is slow at establishing 
benchmarks and the ultimate performance numbers of a fingerprint verification system are 
those which you find in a deployed system. Therefore, one can carry out only a limited 
amount of testing in a laboratory environment to show the anticipated system performance. 
Even in field testing, real performance numbers are not important - it's often the perceived 
performance which is crucial. 

5 Conclusions 

We have designed and implemented an on-line fingerprint verification system which oper- 
ates in two stages: (i) minutia extraction, and (ii) minutia matching. A modified version 
of the minutia extraction algorithm proposed in [18] is used in our system which is much 
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Table 2: Matching rates on test sets using the leave-one-out method: (a) using Identix 
system (180 images); (b) using Digital Biometrics system (610 images). 



Minutia Extraction 
(seconds) 


Minutia Matching 
(seconds) 


Total 
(seconds) 


5.35 


2.55 


7.90 



Table 3: Average CPU time for minutia extraction and matching on a SPARC 20 worksta- 
tion. 

faster and more reliable. A new hierarchical orientation field estimation algorithm results 
in a smoother orientation field which greatly improves the performance of the minutia ex- 
traction. An alignment-based elastic matching algorithm is proposed for minutia matching. 
This algorithm is quite fast, because it is capable of finding the correspondences between 
minutia points without resorting to an exhaustive search. At the same time, this matching 
algorithm has a good performance, because it has the ability to adaptively compensate for 
the nonlinear deformations and inexact pose transformations between different fingerprints. 
Experimental results show that our system achieves excellent performance in a realistic op- 
erating environment. It also meets the response time requirement of on-line verification. 

Based on the experimental results, we observe that the matching errors in our system 
mainly result from (i) incorrect minutiae extraction, and (ii) inaccurate alignment. We 
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observe that a number of factors are detrimental to the correct location of minutia. Among 
them, poor image quality is the most serious one. Therefore, in the future, our efforts will be 
focused on global image enhancement schemes. Another issue related to minutia detection 
is to incorporate a structural-based model in minutia detection which extracts minutiae 
based on their local ridge formations. For elastic matching, an important aspect is to utilize 
additional information (e.g., neighboring ridges) about a minutia to increase the accuracy 
of alignment. 
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Abstract 

Fingerprint matching algorithm is a key issue of the 
fingerprint recognition, and there already exist many 
fingerprint matching algorithms. According to the 
dependence of the core point, fingerprint matching 
algorithms are divided into two groups, core-based match 
algorithms and noncore-based match algorithms. Most of 
the noncore-based matching algorithm is time consuming, 
therefore, they are not suitable for online application; 
meanwhile, the core-based matching algorithm is efficient 
than the noncore-based matching algorithm, but it highly 
depends on the core detection precision. In this paper, we 
present a new core-based structure matching algorithm 
which considers both efficient and precision. Firstly we 
used core detection algorithm to get the core position, 
then we define some local structure of the core area. Used 
these local structure, we can find some correspondent 
points of the two fingerprint image. Secondly, we use the 
correspondent points in the first stage to match the global 
feature of the fingerprint. Experimental results show that 
the performance of the proposed algorithm is good. 



1. Introduction 

Fingerprint-based identification has been used for a 
very long time owning to their uniqueness and 
immutability, today, fingerprints are the most widely used 
biometrics features in automatic verification and 
identification systems. Most Automatic Fingerprint 
Identification Systems (AFIS) are based on local ridge 
feature, such as ridge endings and ridge bifurcation [I]. 

The key issue of the fingerprint recognition is the 
minutiae matching algorithm. Although there already exist 
a lot of matching algorithms, most of them arc not suitable 
for the on line application because of following 
difficulties: 

I. The alignment of two minutiae feature vector. After 
the feature extraction of the fingerprint, we get a minutia 



feature vector. Before match two feature vectors, we need 
to align the two feature vectors. According to usage of the 
core of the fingerprint, the alignment methods can be 
divided into two groups: core-base^ah gnment a nd none 
core-based alignment method. 

2. Due to the noise of the fingerprint image, the 
preprocessing algorithm can not remove all of the noise 
from the feature vector. Therefore, the match algorithm 
should robust to the noise. 

3. The geometric distortion of the fingerprint image 
may greatly affect the match algorithm, and there still is 
no effective method to deal with this nonlinear distortion. 

Moreover, the on-line system require the match speed 
as fast as possible and the template size as small as 
possible, eg. Just the x y coordinates and orient of the 
minutiae in some on line system. In order to solve all of 
the difficulties mentioned above, there are a lot of 
researches dealing with some of them in literature. 

In [12], Z, Chen proposed a topology-based matching 
algorithm of fingerprint authentication. In their algorithm, 
they constructed a structure for minutiae in a pre-specifled 
neighborhood, then they used tree matching algorithm to 
match two fingerprint templates. Their algorithm is 
invariant to translation, rotation and does not depend on 
core point, but their algorithm is greatly affected by noise. 
When the noise increase, the performance of their 
algorithm degrades rapidly. Further more, they 
constructed a structure for each minutiae, this will lead to 
large template. Andrew K Hrechak proposed a structural 
matching algorithm [11], which used the local structure of 
the minutia to describe the characteristics of the minutiae, 
and in [13] some improvements of this approach is 
proposed. Both of those two methods just used the local 
structure information of the fingerprint. However the local 
structure that from same finger may have less similarity 
due to noise, and they need to build the local structure 
during the feature extraction stage, which will lead to 
large size template. Therefore their algorithm is not 
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suitable for. the online application. In [1], Jain proposed 
an alignment-based elastic matching algorithm in which 
the ridge: link to the minutiae is used to alignment the 
feature vector. In order lo do this alignment, they save the 
ten sample-points of the ridge which link to the minutiae. 
This also-leads to large size template. 

In [5],. Xudong. J Proposed a matching algorithm 
which based on local and global structure. First they used 
the k-nearest neighborhood minutiae of each minutia from 
the local! minutia structure, and define a similarity level to 
measure- the similarity of two local structures. Then they 
used the: most similar minutiae pair as the correspondent 
point to- alignment the feature vector, and match the global 
feature, vector considering the local structure similarity. 
This method has some advantages in processing speed and 
robustness to rotation. However, because they need to 
extract the minutiae type and ridge line count between 
each minutiae and its k-nearest neighborhood exactly, 
therefore the performance of their algorithm will be 
greatly affected by the noise. In [6], Nalini K. Ratha 
proposed' a fingerprint authentication using local structural 
algorithm. First they obtained a minimum set of matched 
node pairs by matching their neighborhood structures. 
Then they included more pairs in the match by comparing 
distances with respect to matched pairs obtained in first 
phase. Their methods are robustness in large fingerprint 
database, but their processing speed is low, therefore this 
method is not suitable for the online application. 

When considering an on line application, none of the 
above mentioned method is suitable for this type 
application because of the template size and the system 
efficient. Therefore, in this paper, we propose a new core- 
based structure matching algorithm which consider both 
efficient and precision. Firstly we used core detection 
algorithm to get the core position, and then define some 
local structure for the minutiae near the core point. Used 
those local structure, we find some correspondent point of 
the two fingerprint image. Secondly, we use the 
correspondent points in the first stage to match the global 
feature of the fingerprint. The core point detection method 
is present in section 2. In section 3, the method of match 
local structure is presented and the global feature 
matching algorithm is presented in section 4. In section 5, 
the experiment results of the algorithm are shown. At last, 
the conclusions are given in section 6. 

2. Core point detection 

In early works, we have developed a fast multi- 
resolution based core point detection algorithm [7]. It 
mainly involves two steps: Firstly, we use the low 
resolution direction field to localize a singular area which 
includes core point. Secondly, we use high resolution 
direction field of singular area to precisely localize the 
core point. Used this two step method, we can fast and 
precisely localize the core point in block levels [7]. 



Due to various type of image quality, our algorithm can 
not localize the core point in pixel level; therefore, we can 
not directly use the core point as the reference point for 
the match algorithm. But for most of the case, the core 
point detection is very precise, and if we use this reference 
information, we can greatly reduce the search space, 
therefore, improve the efficient of the match algorithm. In 
this paper, we develop a core based matching algorithm, 
which use the core point to select some minutiae points, 
and use those minutiae points to construct a lot of local 
structure. Correspondence point pair can be got by 
matching those local structures. Use those correspondent 
points pair, the global minutiae matching can be carried 
out. The details of the algorithm are given in section 3. 

3. Local Structure matching 

When an expert recognize a fingerprint, usually he first 
find correspondences of features and then correlate the 
features based on minutia type position , orientation and 
location relative to other features. We usually let the 
computer do as the human expert does. That is, firstly we 
used the local structure of the fingerprint to find some 
correspondence point pairs in local structure matching 
stage, and then we use the correspondent point pairs to 
match the global feature vector of the two fingerprints. 

As we have known, local structure of the fingerprint 
minutiae has some properties which help us to match the 
fingerprint: 

1) Local structures in a small area from the same 
fingerprint are similar in minutiae type, distance and 
angle. 

2) Local structures in a small area that from different 
fingerprint are often different in minutiae type, distance, 
angle and topology information. 

3) Fingerprints which come from the same finger often 
have many similar local structures. 

4) Fingerprints which come from the different finger 
often have little similar local structure than those from the 
same one. 

Usually there are 30-60 minutiae in one fingerprint, if 
we construct local structure for every minutia, the 
template size will be very large, and large template will 
decrease the system's efficient. In our algorithm, we only 
select the minutia which near to the core point of the 
fingerprint, the selection rule is: x 

\M-Core\<R (1) 

Where M and Core denote the minutia and core point 
separately, 1 1 denote the distance function, R is a constant 
which is determine by experiment. Usually, 10-15 
minutia is enough for the local structure. Therefore, reader 
can adjust the R according to the minutiae number. 

3.1 Definition of Local Structure 
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In this section, we present the local structure of the 
minutiae that used in our match algorithm. Usually, there 
are two ways to construct a local structure: 

First, use all of the minutiae around a minutia within 
a limit distance to make a local structure. The problem of 
this method is how to decide the radius, as we have 
known, if the radius is too small, it will include very few 
minutiae, this will lead false match with the noise 
influence. And with large radius, the structure will be 
seriously affected by the elastic distortion. Second, use the 
k-nearest neighborhood of the minutiae to construct a 
local structure. 




Flgurel- A sample of a local structure 

In our match algorithm, we used the second method 
to make a local structure. An example of local structure is 
shown in figure 1 . 

Figure 2 shows an example of four-nearest neighbor 
local structure, where we call the line which connected the 
center minutiae and the neighbor minutiae as edge. Use 
the follow information* we construct local structure: 
Flgure2. Four-nearest neighbor local structure 




1) The distance of the center minutiae and the neighbor 
minutiae, as the D shown in figure 2 

2) The angle (X between center minutiae's orient and 
neighbor minutiae's orient in figure 2. 

3) The angle ft between the minutiae orientation and 
the edge. As shown in figure 2. 

4) The angle Y between the neighboring two edges, as 
shown in figure 2. 

As we have known, the elastic distortion of 
fingerprint in a local small area often similar, so the 
distance between the center minutiae and the neighbor 
minutiae arc invariable to translate and rotation. It is 



apparently that the angle OL , /? , ^ are invariable to 
translate, rotation. Therefore, we can use those four 
features to match the local structure. 

3.2 The local structure matching algorithm 

In this section, we will match the local structure in 
order to find some correspondent point pairs of the two 
fingerprints. 

There arc two methods to match the local structure: 
The first one is to compute the match cost of the two local 
structures. Then select the match pair which has the 
minimal cost as the correspondence pairs. 

The second one is to compute the matching edge of 
the local structure. We can compare the edge from the two 
local structures. If the edge are simitar in distance, angle 
etc, we say those two edge are match. Moreover, if two 
edges of one local structure are match with the other two 
edge of the other local structure, the angles should be 
similar. Through match all edges of the local structure, we 
can get the match edge count, and set the match score of 
the two local structure as the match edge count. 

Through experiment we found that the second method 
are both efficient and robust to noise, therefore we select 
second as our local structure matching algorithm. 

Now wc describe our local structure matching 
algorithm in details. For each edge of the two local 
structures we compute: 

1 ) The relative distance difference R d of the two edges, 

suppose D, and D 2 are distance of two edge's from 
different local structure. 

R^fa-D^/ minify, D } ) (2) 

The min() function means to select the minima! of 
two values. 

2) The angle difference. 

D a =a { -a 2 

If R d , D a , Dp are below a given threshold, we say 

the two edges are matching. 

3) Two edges of the template which match the other 
edges of the input image, we compute the difference 
of the angle Y % denote as D f . 

If the D y is below a given threshold, we said that both 

of those two edges are really match, and cumulate the 
match score of the two local structures with two. 
After those steps, wc will get a match score of the two 
local structures. In order to avoid one edge from a local 
structure match more than one edge with the other local 
structure, we set a flag with the compared edge, and just 
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use the edge which hasn't be matched to start the new 
edge match. 

4. Global matching algorithm 

In section 3, we match the local structure around the 
core point, However, just match the local structure is not 
enough for the verification decision because of the local 
structure usually tend to similar among different 
fingerprints. Therefore in this section, we match the global 
feature based on the local structure match result. From the 
local structure, we select the two structures which have 
maximum match score, and use the center minutiae of the 
two structures as the correspondent point pair of the two 
fingerprints. Using this correspondent point pair, we 
normalize the global feature. 
Let 

pAw.yfA'Y (*:.jtf.«) r ) 

denotes the minutiae feature vector of the template and 

Q=to-yi.w T >-.(*:. y:.w r ) 

denotes the minutiae feature vector of the input image. If 
we denote the reference point &s(x r ,y r ,9 r ) , we have 
the normalization formulation: 



M 




f 






arctan 






V 
0, 



x { - x r 



(3) 



Where the (x ( l J' ( ,# 1 ) denote the minutiae from both the 
template and input image. After the normalization, we 
match the minutiae vector through an elastic box, as 
suggest by Jain ( 1 ]. 

In order to avoid one minutia match many minutiae or 
many minutiae match to one minutia, we construct a 
match table M for each matched minutiae pair /J t Qj and 

we compute the "distance " of those match minutiae pair, 
let 

&a - abs(9 i - 0 J ) + abs^ - o } ) (4) 
And save this "distance" into the match table, 

After the total global match finish, wc search the match 
table, the search rule is: 

For each row, if there exists nonzero value, we get the 
minimum value, and set this column and row to zero 
except this minimum value. After this operation, the 
match number is the nonzero count k in the match table. 
We define a match rate for two fingerprints; 



R = Jk*k/m*n <5) 
Where R is the match rate, and k is the matched pair 
count, m, n is the minutiae count from template and input 
fingerprint image. Through an experience threshold, wc 
can make our decision, that is to say, whether the two 
fingerprints come from a same finger. 

5. Experiment Result and analysis 

In order to evaluate our algorithm, we test our 
algorithm on our fingerprint image database. The image 
size is 300*300 pixels. Our fingerprint image database 
includes two parts. Part 1 is 100 image per finger, 4000 
image altogether. Part II is 20 images per finger, 4000 
image altogether too. We test the FRR on the Part I, and 
test FAR on the Part II. Our fingerprint image database 
includes various type quality fingerprint images, as image 
shown in figure 3, 4, 5. 




Figure3. Good quality fingerprint image 




Flgure4. Middle quality fingerprint image 



We do altogether 80000 times matching. The test 
results are show at table I. The comparison with Jain's 
algorithm [1] is given out in table 3. 



73 




Figures. Poor quality fingerprint image 



Table 1 Fingerprint verification result of our 





False Accept Rate 


False Refuse Rate 


1 


5.0% 


0.08% 


2 


4.0% 


0.09% 


3 


3.0% 


0.35% 


4 


2.0% 


0.58% 


5 


1.0% 


0.80% 


Table 2 Fingerprint verification result of Jain' 
algorithm 




False Accept Rate 


False Refuse Rate 


l 


5.0% 


0.8% 


2 


4.0% 


1.7% 


3 


3.0% 


1.8% 


4 


2.0% 


3.2% 


5 


1.0% 


5.6% 



From the table we can see that, our algorithm has better 
performance in our live fingerprint database. Moreover, 
the average template size is 256 Byte, and the total time 
consuming of our algorithm is 0.001 second on Pill 450M 
Hz. From the test result we can see that our algorithm has 
both better performance and efficient, therefore is more 
suitable for online application. 

The reason that leads to the verification failed is the 
poor quality fingerprint image as shown in figure 5. Those 
types of fingerprint images are confused even by manual 
verification. If we add some strict image quality estimate 
algorithm, the performance can greatly improved, 
therefore the future work of our system is to develop some 
strict image quality estimate algorithm as well as improve 
the minutiae extraction precision. 



out based on those correspondence minutiae. Experiment 
results show that the performance and the efficient are 
significant improved with our algorithm. 
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6. Conclusion 



In this paper, we propose a core based structure 
matching algorithm of fingerprint, which use the minutiae 
near core point of the fingerprint to construct some local 
structures. Through matching the local structures, we can 
get correspondence minutiae. The global match is carried 
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D. Maltoni, D. Maio, A.K. Jain, S. Prabhakar 
Handbook of Fingerprint Recognition 

Springer, New York, 2003 



3.6: Singularity and Core Detection 

(extract) 



(Copyright 2003, Springer Verlag. All rights Reserved.) 
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c) d) 



Figure 3.13. Segmentation of a fingerprint image as proposed by Ratha, Chen, and Jain (1995): 
a) original image; b) variance field; c) quality image derived from the variance field: a quality 
value "good," "medium," "poor" or "background" is assigned to each block according to its vari- 
ance; d) segmented image. ©Elsevier. 



3.6 Singularity and Core Detection 

Most of the approaches proposed in the literature for singularity detection operate on the fin- 
gerprint orientation image. In the rest of this section, the main approaches are coarsely classi- 
fied and a subsection is dedicated to each family of algorithms. 
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Poincare method index 

An elegant and practical method based on the Poincare* index was proposed by Kawagoe and 
Tojo (1984). Let G be a vector field and C be a curve immersed in G; then the Poincare index 
P GtC is defined as the total rotation of the vectors of G along C (see Figure 3.14). 




Figure 3.14. The Poincare index computed over a curve Cimmersed in a vector field G. 



Let G be the field associated with a fingerprint orientation image 1 D and let [ij] be the posi- 
tion of the element & 0 in the orientation image; then the Poincare* index ic.cOV) at [ij] is com- 
puted as follows. 

• The curve C is a closed path defined as an ordered sequence of some elements of D, 
such that [ij] is an internal point; 

• Pc,dij) is computed by algebraically summing the orientation differences between 
adjacent elements of C. Summing orientation differences requires a direction (among 
the two possible) to be associated at each orientation. A solution to this problem is to 
randomly select the direction of the first element and assign the direction closest to 
that of the previous element to each successive element. It is well known and can be 
easily shown that, on closed curves, the Poincare" index assumes only one of the dis- 
crete values: 0°, ±180°, and ±360°. In the case of fingerprint singularities: 

if [/, j] does not belong to any singular region 
if ['f j] belongs to a whorl type singular region 
if ['» j] belongs to a loop type singular region 
if [/, j] belongs to a delta type singular region. 



0° 
360° 
180° 
-180° 



1 Note that a fingerprint orientation image is not a true vector field inasmuch as its elements are unori- 
ented directions. 
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Figure 3.15 shows three portions of orientation images. The path defining C is the ordered 
sequence of the eight elements d k (k = 0.J) surrounding [ij]. The direction of the elements d k 
is chosen as follows: do is directed upward; d* (k= 1..7) is directed so that the absolute value 
of the angle between d k and d k .i is less than or equal to 90°. The Poincare' index is then com- 
puted as 



^c.c('.y)= X^ /e ( d ^ d (^i)mod 8 )- 



k=0..1 
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Figure 3.15. Example of computation of the Poincare index in the 8-neighborhood of points be- 
longing (from the left to the right) to a whorl, loop, and delta singularity, respectively. Note that 
for the loop and delta examples (center and right), the direction of do is first chosen upward (to 
compute the angle between do and di) and then successively downward (when computing the 
angle between 67 and do). 



An example of singularities detected by the above method is shown in Figure 3. 16.a. 

An interesting implementation of the Poincare" method for locating singular points was 
proposed by Bazen and Gerez (2002b); according to Green's theorem, a closed line integral 
over a vector field can be calculated as a surface integral over the rotation of this vector field; 
in practice, instead of summing angle differences along a closed path, the authors compute the 
"rotation" of the orientation image (through a further differentiation) and then perform a local 
integration (sum) in a small neighborhood of each element. Bazen and Gerez (2002b) also 
provided a method for associating an orientation with each singularity; this is done by compar- 
ing the orientation image around each detected singular point with the orientation image of an 
ideal singularity of the same type. 

Singularity detection in noisy or low-quality fingerprints is difficult and the Poincare' 
method may lead to the detection of false singularities (Figure 3.17). Regularizing the orienta- 
tion image through a local averaging, as discussed in Section 3.3, is often quite effective in 
preventing the detection of false singularities. 
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Figure 3.16. Singularity detection by using the Poincare index method. The elements whose 
Poincare index is 180° (loop) or -180° (delta) are enclosed by small boxes. Usually, more than 
one point (four points in these examples) is found for each singular region: hence, the center of 
each singular region can be defined as the barycenter of the corresponding points. 




Figure 3.1 7. a) A poor quality fingerprint; b) the singularities of the fingerprint in a) are extracted 
through the PoincarS method (circles highlight the false singularities); c) the orientation image 
has been regularized and the Poincare method no longer provides false alarms. 
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Registration is a fundamental task in image processing used to match two or more 
pictures taken, for example, at different times, from different sensors, or from different 
viewpoints. Virtually all large systems which evaluate images require the registration 
of images, or a closely related operation, as an intermediate step. Specific examples of 
systems where image registration is a significant component include matching a target 
with a real-time image of a scene for target recognition, monitoring global land usage 
using satellite images, matching stereo images to recover shape for autonomous 
navigation, and aligning images from different medical modalities for diagnosis. 

Over the years, a broad range of techniques has been developed for various types of 
data and problems. These techniques have been independently studied for several 
different applications, resulting in a large body of research. This paper organizes this 
material by establishing the relationship between the variations in the images and the 
type of registration techniques which can most appropriately be applied. Three major 
types of variations are distinguished. The first type are the variations due to the 
differences in acquisition which cause the images to be misaligned. To register images, 
a spatial transformation is found which will remove these variations. The class of 
transformations which must be searched to find the optimal transformation is 
determined by knowledge about the variations of this type. The transformation class in 
turn influences the general technique that should be taken. The second type of 
variations are those which are also due to differences in acquisition, but cannot be 
modeled easily such as lighting and atmospheric conditions. This type usually effects 
intensity values, but they may also be spatial, such as perspective distortions. The 
third type of variations are differences in the images that are of interest such as object 
movements, growths, or other scene changes. Variations of the second and third type 
are not directly removed by registration, but they make registration more difficult 
since an exact match is no longer possible. In particular, it is critical that variations of 
the third type are not removed. Knowledge about the characteristics of each type of 
variation effect the choice of feature space, similarity measure, search space, and 
search strategy which will make up the final technique. All registration techniques can 
be viewed as different combinations of these choices. This framework is useful for 
understanding the merits and relationships between the wide variety of existing 
techniques and for assisting in the selection of the most suitable technique for a 
specific problem. 

Categories and Subject Descriptors: A.l [General Literature]: Introductory and 
Survey; 1.2.10 [Artificial Intelligence]: Vision and Scene Understanding; 1.4 
[Computing Methodologies]: Image Processing; 1.5 [Computing Methodologies]: 
Pattern Recognition 

General Terms: Algorithms, Design, Measurement, Performance 

Additional Key Words and Phrases: Image registration, image warping, rectification, 
template matching 
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1. INTRODUCTION 

A frequent problem arises when images 
taken, at different times, by different 
sensors or from different viewpoints need 
to be compared. The images need to be 
aligned with one another so that differ- 
ences can be detected. A similar problem 
occurs when searching for a prototype or 
template in another image. To find the 
optimal match for the template in the 
image, the proper alignment between the 
image and template must be found. All of 
these problems, and many related varia- 
tions, are solved by methods that per- 
form image registration. A transforma- 
tion must be found so that the points in 
one image can be related to their corre- 
sponding points in the other. The deter- 
mination of the optimal transformation 
for registration depends on the types of 
variations between the images. The ob- 
jective of this paper is to provide a frame- 
work for solving image registration tasks 
and to survey the classical approaches. 

Registration methods can be viewed as 
different combinations of choices for the 
following four components: 

(1) a feature space, 

(2) a search space, 

(3) a search strategy, and 

(4) a similarity metric. 



The feature space extracts the informa- 
tion in the images that will be used for 
matching. The search space is the class 
of transformations that is capable of 
aligning the images. The search strategy 
decides how to choose the next transfor- 
mation from this space, to be tested in 
the search for the optimal transforma- 
tion. The similarity metric determines 
the relative merit for each test. Search 
continues according to the search strat- 
egy until a transformation is found whose 
similarity measure is satisfactory. As we 
shall see, the types of variations present 
in the images will determine the selec- 
tion for each of these components. 

For example, consider the problem of 
registering the two x-ray images of chest 
taken of the same patient at different 
times shown in Figure 1. Properly align- 
ing the two images is useful for detect- 
ing, locating, and measuring pathological 
and other physical changes. A standard 
approach to registration for these images 
might be as follows: the images might 
first be reduced to binary images by de- 
tecting the edges or regions of highest 
contrast using a standard edge detection 
scheme. This removes extraneous infor- 
mation and reduces the amount of data 
to be evaluated. If it is thought that the 
primary difference in acquisition of the 
images was a small translation of the 
scanner, the search space might be a set 
of small translations. For each transla- 
tion of the edges of the left image onto 
the edges of the right image, a measure 
of similarity would be computed. A typi- 
cal similarity measure would be the cor- 
relation between the images. If the simi- 
larity measure is computed for all trans- 
lations then the search strategy is simply 
exhaustive. The images are registered 
using the translation which optimizes the 
similarity criterion. However, the choice 
of using edges for features, translations 
for the search space, exhaustive search 
for the search strategy and correlation 
for the similarity metric will influence 
the outcome of this registration. In fact, 
in this case, the registration will un- 
doubtably be unsatisfactory since the im- 
ages are misaligned in a more complex 
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Figure 6. Any plane can be decomposed into lines parallel to the image plane. 



where A, B, and C are constants, rectifi- 
cation can be performed by mapping the 
intensity of the image point at (jc M y x ) 
into the new rectified image point loca- 
tion (fx t /Z, fyjZ) where Z = f - Ax, - 
By t [Rosenfeld 1982]. This is because the 
scene plane can be decomposed into lines 
Ax Q + By Q = C each at a constant dis- 
tance (z Q = C - C) from the image 
plane. Each line then maps to a line in 
the image plane, and since its perspec- 
tive distortion is related to its distance 
from the image, all points on this line 
must be scaled accordingly by f/(C - C 
- f). Figure 6 shows how a plane is de- 
composed into lines that are each paral- 
lel to the image plane. 

Two pictures of the flat plane taken 
from different viewpoints can be regis- 
tered by the following steps. First, the 
scene coordinates {x x> y 1% z x ) are related 
to their image coordinates in image 1 of a 
point with respect to camera 1 by a scale 
factor (zj - f)/f dependent on their 
depth (the z l coordinate) and the lens 
center f because of similar triangles. This 
gives us two equations. Since they must 
also satisfy the equation of the plane, we 
have three equations from which we can 
derive the three coordinates of each scene 
point using its corresponding image point 



with respect to coordinate system of cam- 
era 1. The scene coordinates are then 
converted from the coordinate system 
with respect to camera 1 to a coordinate 
system with respect to camera 2 to ob- 
tain (x 2 , y 2 , z 2 ). Lastly, these can be pro- 
jected onto image 2 by the factor f/(z 2 - 
f), again by similar triangles. Of course, 
if these are discrete images, there is still 
the problem of interpolation if the regis- 
tered points do not fall on grid locations. 
See Wolberg [1990] for a good survey of 
interpolation methods. 



3. REGISTRATION METHODS 

3.1 Correlation and Sequential Methods 

Cross-correlation is the basic statistical 
approach to registration. If is often used 
for template matching or pattern recog- 
nition in which the location and orienta- 
tion of a template or pattern is found in a 
picture. By itself, cross-correlation is not 
a registration method. It is a similarity 
measure or match metric, i.e., it gives a 
measure of the degree of similarity be- 
tween an image and a template. How- 
ever, there are several registration meth- 
ods for which it is the primary tool, and 
it is these methods and the closely re- 
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lated sequential methods which are dis- 
cussed in this section. These methods are 
generally useful for images which are 
misaligned by small rigid or affine 
transformations. 

For a template T and image 7, where T 
is small compared to 7, the two-dimen- 
sional n^jrjjrjaijje^y^ 
tion measures the similarity for each 
translation: 



C(u,v) = 



L x L y T(x,y)I(x -u y y - v) 
-^[L x L y I 2 (x - u t v - u)] 



malize, as before, using the local image 
energy L x L v I 2 (x - u, y - v). Notice 
that if you expand this intuitive measure 
D(u,v) into its quadratic terms, there 
are three terms: a template energy term, 
a product term of template and image, 
and an image energy term. It is the prod- 
uct term or correlation L x L y T{x, y)I{x 
- u, y - v) which when normalized, de- 
termines the outcome of this measure. 

A related measure, which is advan- 
tageous when an absolute measure is 
needed, is the correlation coefficient 



covariancediT) 



If the template matches the image ex- 
actly, except for an intensity scale factor, 
at a translation of(i } j) t the cross-corre- 
lation will have its peak at C(i } j). (See 
Rosenfeld and Kak [1982] for a proof of 
this using the Cauchy-Schwarz inequal- 
ity.) Thus, by computing C over all pos- 
sible translations, it is possible to find 
the degree of similarity for any 
template-sized window in the image. No- 
tice the cross-correlation must be nor- 
malized since local image intensity would 
otherwise influence the measure. 

The cross-correlation measure is di- 
rectly related to the more intuitive mea- 
sure which computes the sum of the dif- 
ferences squared between the template 
and the picture at each location of the 
template: 



— I(x — u, y — u))~\ 

This measure decreases with the degree 
of similarity since, when the template is 
placed over the picture at the location 
lu,v) for which the template is most simi- 
lar, the differences between the corre- 
sponding intensities will be smallest. The 
template energy defined as L x L y T 2 (x, y) 
is constant for each position ( u , v ) that 
we measure. Therefore, we should nor- 



L x L y (T{x } y) - jLi r )</U - u, y - v) - /i,) 
^L x Jl y {I(x -u,y-v)- fJLjfL^mx.y) - fi T ) 2 

. . where p T and <r T are mean and stan- 
dard deviation of the template and /jlj 
and a l are mean and standard deviation 
of the image. 1 This statistical measure 
has the property that it measures corre- 
lation on an absolute scale ranging from 
[-1,1]. Under certain statistical as- 
sumptions, the value measured by the 
correlation coefficient gives a linear indi- 
cation of the similarity between images. 
This is useful in order to quantitatively 
measure confidence or reliability in a 
match and to reduce the number of mea- 
surements needed when a prespecified 
confidence is sufficient [Svedlow et al. 
1976]. 

Consider a simple example of a binary 
image and binary template, i.e., all the 
pixels are either black or white, for which 
it is possible to predict with some proba- 
bility whether or not a pixel in the image 
will have the same binary value as a 
pixel in the template. Using the correla- 
tion coefficient, it is possible to compute 
the probability or confidence that the im- 
age is an instance of the template. We 
assume the template is an ideal repre- 



1 The mean fi of an image is the average intensity 
value; if the image / is defined over a region jc = 
1, N; jr-l, M then H - Z{* ml L? m {Hx, y)/ 
(N*Ml), The standard deviation is a measure of 
the variation there in the intensity values. It is 
defined as <rf = ^ /( x , y) - 

fi^/iN^M)). 
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sentation of the pattern we are looking 
for. The image may or may not be an 
instance of this pattern. However, if we 
can statistically characterize the noise 
that has corrupted the image, then the 
correlation coefficient can be used to 
quantitatively measure how likely it is 
that the image is an instance of the 
template. 

Another useful property of correlation 
is given by the Correlation theorem. The 
Correlation theorem states that the 
Fourier transform of the correlation of 
two images is the product of the Fourier 
transform of one image and the complex 
conjugate of the Fourier transform of the 
other. This theorem gives an alternate 
way to compute the correlation between 
images. The Fourier transform is simply 
another way to represent the image func- 
tion. Instead of representing the image 
in the spatial domain, as we normally do, 
the Fourier transform represents the 
same information in the frequency do- 
main. Given the information in one do- 
main we can easily convert to the other 
domain. The Fourier transform is widely 
used in many disciplines, both in cases 
where it is of intrinsic interest and as a 
tool, as in this case. It can be computed 
efficiently for images using the Fast 
Fourier Transform or FFT. Hence, an 
important reason why the correlation 
metric is chosen in many registration 
problems is because the Correlation theo- 
rem enables it to be computed efficiently, 
with existing, well-tested programs using 
the FFT (and occasionally in hardware 
using specialized optics). The use of the 
FFT becomes most beneficial for cases 
where the image and template to be 
tested are large. However there are two 
major caveats. Only the cross-correlation 
before normalization may be treated by 
FFT. Second, although the FFT is faster 
it also requires a memory capacity that 
grows with the log of the image area. 
Last, both direct correlation and correla- 
tion using FFT have costs which grow at 
least linearly with the image area. 

Solving registration problems like tem- 
plate matching using correlation has 
many variations [Pratt 1978]. Typically 



the cross-correlation between the image 
and the template (or one of the related 
similarity measures given above) is com- 
puted for each allowable transformation 
of the template. The transformation 
whose cross-correlation is the largest 
specifies how the template can be opti- 
mally registered to the image. This is 
the standard approach when the allow- 
able transformations include a small 
range of translations, rotations, and scale 
changes; the template is translated, ro- 
tated, and scaled for each possible trans- 
lation, rotation, and scale of interest. As 
the number of transformations grows, 
however, the computational costs quickly 
become unmanageable. This is the rea- 
son that the correlation methods are gen- 
erally limited to registration problems in 
which the images are misaligned only by 
a small rigid or affine transformation. In 
addition, to reduce the cost of each meas- 
urement for each transformation in- 
stance, measures are often computed on 
features instead of the whole image area. 
Small local features of the template which 
are more invariant to shape and scale, 
such as edges joined in a Y or a T, are 
frequently used. 

If the image is noisy, i.e., there are 
significant distortions which cannot be 
removed by the transformation, the peak 
of the correlation may not be clearly dis- 
cernible. The Matched Filter Theorem 
states that for certain types of noise such 
as additive white noise, the cross- 
correlation filter that maximizes the ra- 
tio of signal power to the expected noise 
power of the image, i.e., the information 
content, is the template itself. In other 
cases however, the image must be pre- 
filtered before cross-correlation to main- 
tain this property. The prefilter and the 
cross-correlation filter (the template) can 
be used to produce a single filter which 
can simultaneously perform both filter- 
ing operations. The prefilter to be used 
can sometimes be determined if the noise 
in the image satisfies certain statistical 
properties. These techniques, which pre- 
filter based on the properties of the noise 
of the image in order to maximize the 
peak correlation with respect to this noise 
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(using the Matched Filter Theorem) and 
then cross-correlate, are called matched 
filter techniques [Rosenfeld and Kak 
1982]. The disadvantages of these tech- 
niques are that they can be computation- 
ally intensive, and in practice the statis- 
tical assumptions about the noise in the 
image are difficult to satisfy. 

A far more efficient class of algorithms 
than traditional cross-correlation, called 
the sequential similarity detection algo- 
rithms (SSDAs), was proposed by Barnea 
and Silverman [1972]. Two major im- 
provements are offered. First, they sug- 
gest a similarity measure E(u 1 v) ) which 
is computationally much simpler, based 
on the absolute differences between the 
pixels in the two images, 

E(u fV )= E£lru,y) 

x y 

-I(x - u,y - v)\. 
The normalized measure is defined as 

E(u,v)~ LL ] T(x,y)-f 
* y 

-l(x - u, y - v) + I(u,v)\ 

where T and / are the average intensi- 
ties of the template and local image win- 
dow respectively. This is significantly 
more efficient than correlation. Correla- 
tion requires both normalization and the 
added expense of multiplications. Even if 
this measure is unnormalized a mini- 
mum is guaranteed for a perfect match. 
Normalization is useful, however, to get 
an absolute measure of how the two im- 
ages differ, regardless of their intensity 
scales. 

The second improvement Barnea and 
Silverman [1972] introduce is a sequen- 
tial search strategy. In the simplest case 
of translation registration this strategy 
might be a sequential thresholding. For 
each window of the image (determined by 
the translation to be tested and the tem- 
plate size), one of the similarity meas- 
ures defined above is accumulated until 
the threshold is exceeded. For each win- 
dow the number of points that were ex- 
amined before the threshold was ex- 



ceeded is recorded. The window which 
examined the most points is assumed to 
have the lowest measure and is therefore 
the best registration. 

The sequential technique can signifi- 
cantly reduce the computational com- 
plexity with minimal performance degra- 
dation. There are also many variations 
that can be implemented in order to adapt 
the method to a particular set of images 
to be registered. For example, an order- 
ing algorithm can be used to order the 
windows tested which may depend on 
intermediate results, such as a coarse- 
to-fine search or a gradient technique. 
These strategies will be discussed in more 
detail in Section 4.3. The ordering of the 
points examined during each test can also 
vary depending on critical features to be 
tested in the template. The similarity 
measure and the sequential decision al- 
gorithm might vary depending on the re- 
quired accuracy, acceptable speed, and 
complexity of the data. 

Although the sequential methods im- 
prove the efficiency of the similarity 
measure and search, they still have in- 
creasing complexity as the degrees of 
freedom of the transformation is in- 
creased. As the transformation becomes 
more general the size of the search grows. 
On the one hand, sequential search be- 
comes more important in order to main- 
tain reasonable time complexity; on the 
other hand it becomes more difficult not 
to miss good matches. 

In comparison with correlation, the se- 
quential similarity technique improves 
efficiency by orders of magnitude. Tests 
conducted by Barnea and Silverman 
[1972], however, also showed differences 
in results. In satellite imagery taken un- 
der bad weather conditions, clouds 
needed to be detected and replaced with 
random noise before correlation would 
yield a meaningful peak. Whether the 
differences found in their small study 
can be extended to more general cases 
remains to be investigated. 

A limitation of both of these methods is 
their inability to deal with dissimilar im- 
ages. The similarity measures described 
so far, the correlation coefficient, and the 
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sum of absolute differences are maxi- 
mized and minimized, respectively for 
identical matches. For this reason, fea- 
ture-based techniques and measures 
based on the invariant properties of the 
Fourier transform are preferable when 
images are acquired under different cir- 
cumstances, e.g., varying lighting or at- 
mospheric conditions. In the next section 
the Fourier methods will be described. 
Like the correlation and sequential 
methods, the Fourier methods are appro- 
priate for small translations, rotations, 
or scale changes. The correlation meth- 
ods can be used sometimes for more gen- 
eral rigid transformations but become in- 
efficient as the degrees of freedom of the 
transformation grows. The Fourier meth- 
ods can only be used where the Fourier 
transform of an image which has under- 
gone the transformation is related in a 
nice mathematical way to the original 
image. The methods to be described in 
the next section are applicable for images 
which have been translated or rotated or 
both. They are specifically well suited 
for images with low frequency or fre- 
quency-dependent noise; lighting and 
atmospheric variations often cause low- 
frequency distortions. They are not ap- 
propriate for images with frequency-in- 
dependent noise (white noise) or for more 
general transformations. 

3.2 Fourier Method9 

The methods to be described in this sec- 
tion register images by exploiting several 
nice properties of the Fourier Transform. 
Translation, rotation, reflection, distri- 
butivity, and scale all have their counter- 
part in the Fourier domain. Further- 
more, as mentioned in the previous 
section, the transform can be efficiently 
implemented in either hardware or using 
the Fast Fourier Transform. These meth- 
ods differ from the methods in the last 
section because they search for the opti- 
mal match according to information in 
the frequency domain. The method de- 
scribed in Section 3.1 used the Fourier 
Transform as a tool to perform a spatial 
operation, namely correlation. 



By using the frequency domain, the 
Fourier methods achieve excellent ro- 
bustness against correlated and fre- 
quency-dependent noise. They are appli- 
cable, however, only for images which 
have been at most rigidly misaligned. In 
this section we will first describe the most 
basic method which uses Fourier Analy- 
sis. It is called phase correlation and can 
be used to register images which have 
been shifted relative to each other. Then 
we will describe an extension to this 
method and several related methods 
which handle images which have been 
both shifted and rotated with respect to 
each other. 

Kuglin and Hines [1975] proposed an 
elegant method, called phase correlation, 
to align two images which are shifted 
relative to one another. In order to de- 
scribe their method, we will define a few 
of the terms used in Fourier Analysis 
which we will need. The Fourier trans- 
form of an image fix f y) is a complex 
function; each function value has a real 
part R(<o xt co y ) and an imaginary part 
I(o> x > <o y ) at each frequency (a> x , a> y ) of 
the frequency spectrum: 

where i — V~ 1 . This can be expressed 
alternatively using the exponential form 
as 

F{a> x ,<o y ) = \F(a> xi a> y )\e^>*>>> 

where \F(a> x , w y )\ is the magnitude or 
amplitude of the Fourier transform and 
where <f>(<o x , (o y ) is the phase angle. The 
square of the magnitude is equal to the 
amount of energy or power at each fre- 
quency of the image and is defined as: 

\ F {o>x> w „v)| 2 =r2 {°> x > Vy) + /2 ( w *> <*> y )- 

The phase angle describes the amount of 
phase shift at each frequency and is de- 
fined as 

<t>((D xi o> v ) = 

tan-^/C^,^)/^^,^)]. 

Phase correlation relies on the transla- 
tion property of the Fourier transform, 
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